Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.
Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag
The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.
Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl
transparent layering and better fragmentation.
- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.
Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division
* Make Yarrow an optional kernel component -- enabled by "YARROW_RNG" option.
The files sha2.c, hash.c, randomdev_soft.c and yarrow.c comprise yarrow.
* random(4) device doesn't really depend on rijndael-*. Yarrow, however, does.
* Add random_adaptors.[ch] which is basically a store of random_adaptor's.
random_adaptor is basically an adapter that plugs in to random(4).
random_adaptor can only be plugged in to random(4) very early in bootup.
Unplugging random_adaptor from random(4) is not supported, and is probably a
bad idea anyway, due to potential loss of entropy pools.
We currently have 3 random_adaptors:
+ yarrow
+ rdrand (ivy.c)
+ nehemeiah
* Remove platform dependent logic from probe.c, and move it into
corresponding registration routines of each random_adaptor provider.
probe.c doesn't do anything other than picking a specific random_adaptor
from a list of registered ones.
* If the kernel doesn't have any random_adaptor adapters present then the
creation of /dev/random is postponed until next random_adaptor is kldload'ed.
* Fix randomdev_soft.c to refer to its own random_adaptor, instead of a
system wide one.
Submitted by: arthurmesh@gmail.com, obrien
Obtained from: Juniper Networks
Reviewed by: obrien
DB120 development board.
The AR934x SoCs are a MIPS74k based system with increased RAM addressing
space, some scratch-pad RAM, an improved gige switch PHY and 2x2 or 3x3
on-board dual-band wifi.
This support isn't complete by any stretch; it's just enough to bring
the board up for others to tinker with. Notably, the MIPS74k support
is broken. However it boots enough to echo some basic probe/attach
messages, before dying somewhere in the TLB code.
Thankyou to Qualcomm Atheros for their continued support of me doing
open source work with their hardware.
Tested:
* AR9344, mips74k
This code reads the PLL configuration registers and correctly programs
things so the UART and such can come up.
There's MIPS74k platform issues that need fixing; but this at least brings
things up enough to echo stuff out the serial port and allow for interactive
debugging with ddb.
Tested:
* AR71xx SoCs
* AR933x SoC
* AR9344 board (DB120)
Obtained from: Qualcomm Atheros; Linux/OpenWRT
For all pre-AR933x chips, the frequency is just the APB frequency.
For the AR933x, the UART frequency is different but we just hacked around
it.
For the AR934x, there's a different PLL setting for these, so they have
to be broken out.
They originated in the original Octeon port. They weren't present, as
far as I can tell, on the projects/mips branch until after this
point. They were in the original Octeon port in code picked up from
the vendor, who I've been able to find out trolling old email put them
there to get around an SMP problem that most likely was fixed in other
ways.
NetBSD and Linux don't have these, except for some specific uses of
SYNC on the alchemy parts (which we don't support, but even if we did
it is only a specific case and would be specifically coded
anyway). This is true of the current Linux code, as well as one old
version I polled.
I looked back at the old R12000, R8000, R6000, R4000, R4400 errata
that I have, and could find no mention of SYNC needing NOPs for
silicon bugs (although plenty of other cases where NOPs and other
contortions were needed).
An Google search turned up no old mailing list discussions on this on
Linux, NetBSD or FreeBSD (except the disussion that kicked off these
studies).
I've test booted this on my Octeon Plus eval board and survived a
buildworld. Adrian Chadd reports that this patch has no ill effects on
the Ahteros platforms he tested it on.
I conclude it is safe to just remove the NOPs. But added
__MIPS_PLATFORM_SYNC_NOPS as a failsafe in case we find some platform
where these are, in fact, required.
Reviewed by: adrian@
This is an AR7240 based device with an AR9285 on-board.
I've tested the initial boot and wifi support; however at the moment
the ethernet switch driver doesn't seem to be picking up carrier on the
active ethernet port. Basic flood pinging works however, so I think
we're on the right track.
Thank you to Adrian Woodley <adrian@diskworld.com.au> for purchasing me
one of these devices to bootstrap FreeBSD-HEAD on.
Issues were noted by Bruce Evans and are present on all architectures.
On i386, a counter fetch should use atomic read of 64bit value,
otherwise carry from the increment on other CPU could be lost for the
given fetch, making error of 2^32. If 64bit read (cmpxchg8b) is not
available on the machine, it cannot be SMP and it is enough to disable
preemption around read to avoid the split read.
On x86 the counter increment is not atomic on purpose, which makes it
possible for the store of the incremented result to override just
zeroed per-cpu slot. The effect would be a counter going off by
arbitrary value after zeroing. Perform the counter zeroing on the
same processor which does the increments, making the operations
mutually exclusive. On i386, same as for the fetching, if the
cmpxchg8b is not available, machine is not SMP and we disable
preemption for zeroing.
PowerPC64 is treated the same as amd64.
For other architectures, the changes made to allow the compilation to
succeed, without fixing the issues with zeroing or fetching. It
should be possible to handle them by using the 64bit loads and stores
atomic WRT preemption (assuming the architectures also converted from
using critical sections to proper asm). If architecture does not
provide the facility, using global (spin) mutex would be non-optimal
but working solution.
Noted by: bde
Sponsored by: The FreeBSD Foundation
Provide both __sync_*-style and __atomic_*-style functions that perform
the atomic operations on ARMv5 by using Restartable Atomic Sequences.
While there, clean up some pieces of code where it's sufficient to use
regular uint32_t to store register contents and don't need full reg_t's.
Also sync this back to the MIPS code.
After pushing in my fix for the 2 byte functions, I realized that the
functions for 1 and 2 byte operations had become identical. Reduce the
code size by merging the functions for 1 and 2 byte operations together.
While there, slightly improve variable naming and comments.
This is based on the AR933x (Hornet) SoC from Qualcomm Atheros.
It's a much nicer board to do development on - 64MB RAM, 16MB flash.
The development board breaks out the GPIO pins, ethernet, serial (via
a USB<->RS232 chip), USB host and of course a small wifi antenna.
Everything but the wifi works thus far.
Even though I tested the 1-byte operations on arbitrarily aligned bytes,
it seems I did not do this for the 2-byte operations.
Create easy to read functions that are used to get/put bytes and
halfwords in words. To keep the compiler happy, explicitly read two
bytes into a union to obtain a 16-bit value.
To make <stdatomic.h> work on MIPS (and ARM) using GCC, we need to
provide implementations of the __sync_*() functions. I already added
these functions for 4 and 8 byte types to libcompiler-rt some time ago,
based on top of <machine/atomic.h>.
Unfortunately, <machine/atomic.h> only provides a subset of the features
needed to implement <stdatomic.h>. This means that in some cases we had
to do compare-and-exchange calls in loops, where a simple ll/sc would
suffice.
Also implement these functions for 1 and 2 byte types. MIPS only
provides ll/sc instructions for 4 and 8 byte types, but this is of
course no limitation. We can simply load 4 bytes and use some bitmask
tricks to modify only the bytes affected.
Discussed on: mips, arch
Tested with: QEMU
this file is in FreeBSD. There's formality to this that hasn't
happened and Juniper is perfectly fine with being the holder.
Discussed with: eadler, imp, jhb
ownership to the FreeBSD foundation for the years this file has been in
the FreeBSD repository.
This file was originally created by Juniper as part of upgrading to FreeBSD
4.10 (which had no MIPS support) and held functions found on other machines
It grew actual functionality over time. The functionaliy was copied from
other architectures and ported to MIPS on a as-needed basis.
Approved by: Mark Baushke (Juniper IP)
Approved by: Megan Sugiyama (Juniper legal)
Pointed out by: jmallett@
Requested by: core (jhb@)
o Relax locking assertions for pmap_enter_object() and add them also
to architectures that currently don't have any
o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade
operation on the per-object rwlock
o Use all the mechanisms above to make vm_map_pmap_enter() to work
mostl of the times only with readlocks.
Sponsored by: EMC / Isilon storage division
Reviewed by: alc
Convert the structures to C99 style initialisation, which makes it
a lot easier to check that all of them are set and to generate a
derived template from them.
Sponsored by: DARPA, AFRL
MFC after: 2 weeks
Until an ADM6996 driver shows up, this allows for the two switch
ports to be used.
Submitted by: Luiz Otavio O Souza <loos.br@gmail.com>
Reviewed by: ray
order to match the MAXCPU concept. The change should also be useful
for consolidation and consistency.
Sponsored by: EMC / Isilon storage division
Obtained from: jeff
Reviewed by: alc
Remove #define to get kludges that asm.h used to define
Move clever macros to access assembler instructions to trap.c
Remove __ASSEMBLER__ ifdefs in regdef.h: they aren't needed anymore.
expand the %sccs.include.redist.c% directive with the standard
3-clause license, and add $FreeBSD$ to keep the commit script happy.
# This may break some mips stuff, which will be fixed in the next commit.
I/O clock. Thankfully, the simple executive provies a way to querry
the proper clock that works on all models. Move to asking for the SCLK
via this interface.
This gets the serial console working after we start init and open the
console and set the divisor (which turned the output from good to
bad). I can login on the console now.
in the pcb. setjmp/longjmp in the kernel also used these values, so
continue to use them although their use isn't technically the pcb
register array (matching is all that's important for setjmp/longjmp in
the kernel). Finally, eliminate the old register names from regnum.h.
This is a lexical change only. The non-debug .o files have the same md5.
Partially implement generic_bs_*_8() for MIPS platforms.
This is known to work with TARGET_ARCH=mips64 with FreeBSD/BERI.
Assuming that other definitions in cpufunc.h are correct it will
work on non-o64 ABI systems except sibyte. On sibyte and o32 systems
generic_bs_*_8() will remain panic() implementations.
Sponsored by: DARPA, AFRL
Reviewed by: imp, jmallett (older versions)
(Wasting 4k just as a temporary placeholder for a boot environment seems
a bit ridiculous, but hey.)
Tested: gxemul:
$ gxemul -e malta -d i:/home/adrian/work/freebsd/svn/mfsroot-rspro.img -C 4Kc /tftpboot/kernel.MALTA
to unique values.
There's some confusion about what the n32 assembler API really is
(since on page 9 of the spec they say that t0-t3 don't exist, then
turn around on page 22 and say that t4-t7 don't exist), and this
doesn't touch that.
NetBSD's version of this file follows the convention I used here, and
is likely to be correct.
This should fix gdb/ptrace.
sys/arm and sys/mips), squelching the clang 3.3 warnings about this.
Noticed by: tinderbox and many irate spectators
Submitted by: Luiz Otavio O Souza <loos.br@gmail.com>
PR: kern/177759
MFC after: 3 days
Having MIPS_MAX_TLB_ENTRIES defined to 128 is misleading, since it used
to be 64 in older releases of MIPS architecture (where it could be read
from Config1) and can be much more than 128 for the newer processors.
For now, move the definition to the only file using it (mips/mips/tlb.c)
and define MIPS_MAX_TLB_ENTRIES depending on the MIPS cpu defined. Also
add few checks so that we do not write beyond the end of the tlb_state
array.
This fixes a kernel data corruption seen in Netlogic XLP, which was casued
by tlb_save() writing beyond the end of tlb_state array when breaking into
debugger.
Introduce counter(9) API, that implements fast and raceless counters,
provided (but not limited to) for gathering of statistical data.
See http://lists.freebsd.org/pipermail/freebsd-arch/2013-April/014204.html
for more details.
In collaboration with: kib
Reviewed by: luigi
Tested by: ae, ray
Sponsored by: Nginx, Inc.
* arge0 is MII
* arge1 is GMII
* the MDIO bus is on arge1, not arge0
* the default switch config is to have ports 0-3 as the switch group,
with port 4 being an external PHY dedicated to arge0 (ie, 'cpu' port.)
Whilst I'm here, remove unused bits and pieces from the config.
Tested:
* AP121, ping on both arge0 and arge1
* Tested switch port detection using etherswitchcfg
* Enable RX and host interrupts during bus probe/attach
* Disable all interrupts (+ host ISR) during bus detach
* Enable TX DONE interrupt only when we start transmitting; clear it when
we're done.
* The RX/TX FIFO depth is still conjecture on my part. I'll fix this
shortly.
* The TX FIFO interrupt isn't an "empty" interrupt, it's an "almost empty"
interrupt. Sigh. So..
* .. in ar933x_bus_transmit(), wait for the FIFO to drain before
continuing.
I dislike having to wait for the FIFO to drain, alas.
Tested:
* Atheros AP121 board, AR9331 SoC.
TODO:
* RX/TX overflow, RX error, BREAK support, etc.
* Figure out the true RX/TX FIFO depth.
This implements the bus transmit/receive/sigchg/ipend methods with
a polled interrupt handler (ipend) rather than enabling hardware
interrupts.
The FIFO is faked at 16 bytes deep for now, just so the transmit
IO side doesn't suck too bad (the callout frequency limits how quickly
IO is flushed to the sender, rather than scheduling the callout more
frequently whilst there's active TX. But I digress.)
Tested:
* Atheros AP121 (AR9330) reference board, booting to multi-user interactive
mode.
most kernels before FreeBSD 9.0. Remove such modules and respective kernel
options: atadisk, ataraid, atapicd, atapifd, atapist, atapicam. Remove the
atacontrol utility and some man pages. Remove useless now options ATA_CAM.
No objections: current@, stable@
MFC after: never
uart(4) allocates send and receiver buffers in attach() before it calls
the low-level driver's attach routine. Many low-level drivers set the
fifo sizes in their attach routine, which is too late. Other drivers set
them in the probe() routine, so that they're available when uart(4)
allocates buffers. This fixes the ones that were setting the values too
late by moving the code to probe().
* Add baud rate and divisor programming code. See below for more
information.
* Flesh out ar933x_init() to disable interrupts and program the initial
console setup.
* Remove #if 0'ed code from ar933x_term().
* Explain what these functions do.
Now, the baud rate and divisor code comes from Linux, as a submission
to the OpenWRT project and Linux kernel from
Gabor Juhos <juhosg@openwrt.org>.
The original ticket for this code is https://dev.openwrt.org/ticket/12031 .
I've contacted Gabor and asked for his permission to also licence the patch
in question (which covers this code) to BSD lience and he's agreed.
Hence why I'm including it here in FreeBSD.
Tested:
* AP121 (AR9330)
* Default clock is 25MHz;
* Remove the UART register macro here - it's not needed as we don't need
to "adjust" the register offset / spacing at all;
* Remove unused fields in the softc.
Tested:
* AP121
This implements the kernel glue needed (getc, putc, rxready).
This isn't a 16550 UART, even if the datasheet overview claims so.
The Linux ar933x support was used as a reference, however the uart code
is a reimplementation.
Attentive viewers will note that the uart code is based off of the ns8250
code and the UART bus code is a stubbed-out version of this. I'll be
replacing it with non-stubbed versions soon, making this a fully featured
driver.
Tested:
* AP121 reference board (AR933x), booting through the mountroot> prompt;
then doing some basic interactive tests in ddb.
This was ported from the AR724x code and I think that also doesn't
quite work. I'll investigate that soon.
With this in place the system reset path works, so 'reset' from kdb
actually resets the SoC.
Tested:
* AP121 test board
CPUs.
The AR933x is a mips24k based SoC with an AR9380 series SoC on board,
two gigabit ethernet interfaces and an internal 10/100mbit ethernet
switch. There's also the normal interfaces (USB, ethernet, uart, GPIO.)
The downside? There's a non-ns8250 UART device.
With a very basic UART driver (not in this commit) the SoC is initialised
and boots up. I'll commit the UART code soon and then link it into the
general setup path.
This code is a re-implementation based from the Linux kernel / openwrt
AR933x support.
TODO:
* UART (obviously)
* All of the ethernet, USB and wifi SoC glue, including ethernet PLL
programming.
do not map the b_pages pages into buffer_map KVA. The use of the
unmapped buffers eliminate the need to perform TLB shootdown for
mapping on the buffer creation and reuse, greatly reducing the amount
of IPIs for shootdown on big-SMP machines and eliminating up to 25-30%
of the system time on i/o intensive workloads.
The unmapped buffer should be explicitely requested by the GB_UNMAPPED
flag by the consumer. For unmapped buffer, no KVA reservation is
performed at all. The consumer might request unmapped buffer which
does have a KVA reserve, to manually map it without recursing into
buffer cache and blocking, with the GB_KVAALLOC flag.
When the mapped buffer is requested and unmapped buffer already
exists, the cache performs an upgrade, possibly reusing the KVA
reservation.
Unmapped buffer is translated into unmapped bio in g_vfs_strategy().
Unmapped bio carry a pointer to the vm_page_t array, offset and length
instead of the data pointer. The provider which processes the bio
should explicitely specify a readiness to accept unmapped bio,
otherwise g_down geom thread performs the transient upgrade of the bio
request by mapping the pages into the new bio_transient_map KVA
submap.
The bio_transient_map submap claims up to 10% of the buffer map, and
the total buffer_map + bio_transient_map KVA usage stays the
same. Still, it could be manually tuned by kern.bio_transient_maxcnt
tunable, in the units of the transient mappings. Eventually, the
bio_transient_map could be removed after all geom classes and drivers
can accept unmapped i/o requests.
Unmapped support can be turned off by the vfs.unmapped_buf_allowed
tunable, disabling which makes the buffer (or cluster) creation
requests to ignore GB_UNMAPPED and GB_KVAALLOC flags. Unmapped
buffers are only enabled by default on the architectures where
pmap_copy_page() was implemented and tested.
In the rework, filesystem metadata is not the subject to maxbufspace
limit anymore. Since the metadata buffers are always mapped, the
buffers still have to fit into the buffer map, which provides a
reasonable (but practically unreachable) upper bound on it. The
non-metadata buffer allocations, both mapped and unmapped, is
accounted against maxbufspace, as before. Effectively, this means that
the maxbufspace is forced on mapped and unmapped buffers separately.
The pre-patch bufspace limiting code did not worked, because
buffer_map fragmentation does not allow the limit to be reached.
By Jeff Roberson request, the getnewbuf() function was split into
smaller single-purpose functions.
Sponsored by: The FreeBSD Foundation
Discussed with: jeff (previous version)
Tested by: pho, scottl (previous version), jhb, bf
MFC after: 2 weeks
pages around, taking array of vm_page_t both for source and
destination. Starting offsets and total transfer size are specified.
The function implements optimal algorithm for copying using the
platform-specific optimizations. For instance, on the architectures
were the direct map is available, no transient mappings are created,
for i386 the per-cpu ephemeral page frame is used. The code was
typically borrowed from the pmap_copy_page() for the same
architecture.
Only i386/amd64, powerpc aim and arm/arm-v6 implementations were
tested at the time of commit. High-level code, not committed yet to
the tree, ensures that the use of the function is only allowed after
explicit enablement.
For sparc64, the existing code has known issues and a stab is added
instead, to allow the kernel linking.
Sponsored by: The FreeBSD Foundation
Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6)
MFC after: 2 weeks
future further optimizations where the vm_object lock will be held
in read mode most of the time the page cache resident pool of pages
are accessed for reading purposes.
The change is mostly mechanical but few notes are reported:
* The KPI changes as follow:
- VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK()
- VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK()
- VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK()
- VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED()
(in order to avoid visibility of implementation details)
- The read-mode operations are added:
VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(),
VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED()
* The vm/vm_pager.h namespace pollution avoidance (forcing requiring
sys/mutex.h in consumers directly to cater its inlining functions
using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h
consumers now must include also sys/rwlock.h.
* zfs requires a quite convoluted fix to include FreeBSD rwlocks into
the compat layer because the name clash between FreeBSD and solaris
versions must be avoided.
At this purpose zfs redefines the vm_object locking functions
directly, isolating the FreeBSD components in specific compat stubs.
The KPI results heavilly broken by this commit. Thirdy part ports must
be updated accordingly (I can think off-hand of VirtualBox, for example).
Sponsored by: EMC / Isilon storage division
Reviewed by: jeff
Reviewed by: pjd (ZFS specific review)
Discussed with: alc
Tested by: pho
Switch eventtimers(9) from using struct bintime to sbintime_t.
Even before this not a single driver really supported full dynamic range of
struct bintime even in theory, not speaking about practical inexpediency.
This change legitimates the status quo and cleans up the code.
every architecture's busdma_machdep.c. It is done by unifying the
bus_dmamap_load_buffer() routines so that they may be called from MI
code. The MD busdma is then given a chance to do any final processing
in the complete() callback.
The cam changes unify the bus_dmamap_load* handling in cam drivers.
The arm and mips implementations are updated to track virtual
addresses for sync(). Previously this was done in a type specific
way. Now it is done in a generic way by recording the list of
virtuals in the map.
Submitted by: jeff (sponsored by EMC/Isilon)
Reviewed by: kan (previous version), scottl,
mjacob (isp(4), no objections for target mode changes)
Discussed with: ian (arm changes)
Tested by: marius (sparc64), mips (jmallet), isci(4) on x86 (jharris),
amd64 (Fabian Keil <freebsd-listen@fabiankeil.de>)