In addition to some small style fixes to the ARMv6 vDSO, this release
includes a new vDSO that can be used for the execution of ARMv6/ARMv7
code on 64-bit platforms.
Just like for i686 on x86-64, this new vDSO is responsible for padding
arguments and return values to 64-bit values, so that the kernel can
easily forward system calls to the native system calls.
Obtained from: https://github.com/NuxiNL/cloudabi
link-local addresses when VIMAGE is enabled will cause a so-called NULL
pointer dereferencing issue.
Sponsored by: Mellanox Technologies
MFC after: 1 week
The driver was printing out a lot of information upon failure, which
does not have to be interested for the user.
Changing logging level required to rebuild driver with proper flags. The
proper sysctl was added, so the level now can be changed dynamically
using bitmask.
Levels of printouts were adjusted to keep on mind end user instead of
debugging purposes.
More verbose messages were added to align the driver with the Linux.
Fix building error introduced by the r325506 by casting csum_flags to
uint64_t.
Submitted by: Michal Krawczyk <mk@semihalf.com>
Reviewed by: byenduri_gmail.com
Obtained from: Semihalf
Sponsored by: Amazon, Inc.
Differential Revision: https://reviews.freebsd.org/D12868
Compared to the previous version, v0.16, there are a couple of minor
changes:
- CLOUDABI_AT_PID: Process identifiers for CloudABI processes.
Initially, BSD process identifiers weren't exposed inside the runtime,
due to them being pretty much useless inside of a cluster computing
environment. When jobs are scheduled across systems, the BSD process
number doesn't act as an identifier. Even on individual systems they
may recycle relatively quickly.
With this change, the kernel will now generate a UUIDv4 when executing
a process. These UUIDs can be obtained within the process using
program_getpid(). Right now, FreeBSD will not attempt to store this
value. This should of course happen at some point in time, so that it
may be printed by administration tools.
- Removal of some unused structure members for polling.
With the polling framework being simplified/redesigned, it turns out
some of the structure fields were not used by the C library. We can
remove these to keep things nice and tidy.
Obtained from: https://github.com/NuxiNL/cloudabi
The newest ena-com HAL supports LLQv2 and introduces
API changes. In order not to break the driver compilation
it was updated/fixed in a following way:
* Change version of the driver to 0.8.0
* Provide reset cause when triggering reset of the device
* Reset device after attach fails
* In the reset task free management irq after calling ena_down. Admin
queue can still be used before ena_down is called, or when it is
being handled
* Do not reset device if ena_reset_task fails
* Move call of the ena_com_dev_reset to the ena_down() routine - it
should be called only if interface was up
* Use different function for checking empty space on the sq ring
(ena-com API change)
* Fix typo on ENA_TX_CLEANUP_THRESHOLD
* Change checking for EPERM with EOPNOTSUPP - change in the ena-com API
* Minor style fixes
Submitted by: Michal Krawczyk <mk@semihalf.com>
Obtained from: Amazon.com, Inc.
Semihalf
Sponsored by: Amazon.com, Inc.
Differential Revision: https://reviews.freebsd.org/D12143
Rework the dTSEC and FMan drivers to be more like a full bus relationship,
so that dtsec can use bus_alloc_resource() instead of trying to handle the
offset from the dts. This required taking some code from the sparc64 ebus
driver to allow subdividing the fman region for the dTSEC devices.
This adds some support for ARM as well as 64-bit. 64-bit on PowerPC is
currently not working, and ARM support has not been completed or tested on the
FreeBSD side.
As this was imported from a Linux tree, it includes some Linux-isms
(ioread/iowrite), so compile with the LinuxKPI for now. This may change in the
future.
In rS323851, some casts were adjusted in calls to nvlist_next() and
nvlist_get_pararr() in order to make scan-build happy. I think these changes
just confused scan-build into not reporting the strict-aliasing violation.
For example, nvlist_xdescriptors() is causing nvlist_next() to write to its
local variable nvp of type nvpair_t * using the lvalue *cookiep of type
void *, which is not allowed. Given the APIs of nvlist_next(),
nvlist_get_parent() and nvlist_get_pararr(), one possible fix is to create a
local void *cookie in nvlist_xdescriptors() and other places, and to convert
the value to nvpair_t * when necessary. This patch implements that fix.
Reviewed by: oshogbo
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D12760
The most important change in this release is the removal of the
poll_fd() system call; CloudABI's equivalent of kevent(). Though I think
that kqueue is a lot saner than many of its alternatives, our
experience is that emulating this system call on other systems
accurately isn't easy. It has become a complex API, even though I'm not
convinced this complexity is needed. This is why we've decided to take a
different approach, by looking one layer up.
We're currently adding an event loop to CloudABI's C library that is API
compatible with libuv (except when incompatible with Capsicum).
Initially, this event loop will be built on top of plain inefficient
poll() calls. Only after this is finished, we'll work our way backwards
and design a new set of system calls to optimize it.
Interesting challenges will include integrating asynchronous I/O into
such a system call API. libuv currently doesn't aio(4) on Linux/BSD, due
to it being unreliable and having undesired semantics.
Obtained from: https://github.com/NuxiNL/cloudabi
initialize nvp on every loop iteration and the code under 'fail'(!) label
detects success by checking of nvp != NULL.
Submitted by: pjd@
MFC after: 1 month
Sponsored by: Wheel Systems
to 'fail' on error it was treated as success, because nvp!=NULL. Fix this
by not handling success under 'fail' label and by using separate variable
for parent nvpair.
If we succeeded to allocate nvlist, but failed to allocated nvpair we
would leak nvls[ii] on return. Destroy it when we cannot allocate nvpair,
before we goto fail.
Submitted by: pjd@ and oshogbo@ (minor changes)
Found by: scan-build
MFC after: 1 month
Sponsored by: Wheel Systems
initially NULL, which is not possible. Change the loop to
'do {} while (array != NULL)' to satisfy scan-build and assert that
array really cannot be NULL just in case.
Submitted by: pjd@
Found by: scan-build
MFC after: 1 month
Sponsored by: Wheel Systems
Make scan-build happy by casting to 'void *' instead of 'void **'.
Submitted by: pjd@
MFC after: 1 month
Found by: scan-build and cppcheck
Sponsored by: Wheel Systems
Now that CloudABI's sockets API has been changed to be addressless and
only connected socket instances are used (e.g., socket pairs), they have
become fairly similar to pipes. The only differences on CloudABI is that
socket pairs additionally support shutdown(), send() and recv().
To simplify the ABI, we've therefore decided to remove pipes as a
separate file descriptor type and just let pipe() return a socket pair
of type SOCK_STREAM. S_ISFIFO() and S_ISSOCK() are now defined
identically.
Now that all of the packaged software has been adjusted to either use
Flower (https://github.com/NuxiNL/flower) for making incoming/outgoing
network connections or can have connections injected, there is no longer
need to keep accept() around. It is now a lot easier to write networked
services that are address family independent, dual-stack, testable, etc.
Remove all of the bits related to accept(), but also to
getsockopt(SO_ACCEPTCONN).
With Flower (CloudABI's network connection daemon) becoming more
complete, there is no longer any need for creating any unconnected
sockets. Socket pairs in combination with file descriptor passing is all
that is necessary, as that is what is used by Flower to pass network
connections from the public internet to listening processes.
Remove all of the kernel bits that were used to implement socket(),
listen(), bindat() and connectat(). In principle, accept() and
SO_ACCEPTCONN may also be removed, but there are still some consumers
left.
Obtained from: https://github.com/NuxiNL/cloudabi
MFC after: 1 month
The CloudABI specification has had some minor changes over the last half
year. No substantial features have been added, but some features that
are deemed unnecessary in retrospect have been removed:
- mlock()/munlock():
These calls tend to be used for two different purposes: real-time
support and handling of sensitive (cryptographic) material that
shouldn't end up in swap. The former use case is out of scope for
CloudABI. The latter may also be handled by encrypting swap.
Removing this has the advantage that we no longer need to worry about
having resource limits put in place.
- SOCK_SEQPACKET:
Support for SOCK_SEQPACKET is rather inconsistent across various
operating systems. Some operating systems supported by CloudABI (e.g.,
macOS) don't support it at all. Considering that they are rarely used,
remove support for the time being.
- getsockname(), getpeername(), etc.:
A shortcoming of the sockets API is that it doesn't allow you to
create socket(pair)s, having fake socket addresses associated with
them. This makes it harder to test applications or transparently
forward (proxy) connections to them.
With CloudABI, we're slowly moving networking connectivity into a
separate daemon called Flower. In addition to passing around socket
file descriptors, this daemon provides address information in the form
of arbitrary string labels. There is thus no longer any need for
requesting socket address information from the kernel itself.
This change also updates consumers of the generated code accordingly.
Even though system calls end up getting renumbered, this won't cause any
problems in practice. CloudABI programs always call into the kernel
through a kernel-supplied vDSO that has the numbers updated as well.
Obtained from: https://github.com/NuxiNL/cloudabi
list.h includes a number of FreeBSD headers as a workaround for the
LIST_HEAD name collision. To reduce pollution, avoid including list.h
in commonly used headers when it is not explicitly needed.
Reviewed by: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D11249
This patch currently supports:
- ibcore as a kernel module only
- krping as a kernel module only
- ipoib as a kernel module only
Sponsored by: Mellanox Technologies
It turns out that this is useful on hornet and wasp SoCs but it isn't
enabled in ye olde HAL /unless/ you were using a version from one of the
business units building USB targetted devices. It eventually got fixed
for all of them as people started wanting to use the USB ports on their
SoCs (eg for flash storage, bluetooth, 4G/LTE widgets, etc.)
This is actually a fix from ath9k but I'm merging it with the available-but-
disabled code in the QCA reference HAL.
Tested:
* AR9331 SoC
Summary:
The Ubiquiti Unifi Security Gateway is virtually identical to the
EdgeRouter Lite, with a smaller PCB and apparently a different board identifier.
Simply adding the new board identifier alongside the ERL identifier, FreeBSD
boots successfully, and can access the needed peripherals (tested with USB
booting, and basic pings on one ethernet interface)
Reviewed By: adrian
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D10639
ipfilter man pages. This also currently restricts keep frags to only when
keep state is used, which is redundant because keep state currently
assumes keep frags. This commit fixes this.
To the user this change means that to maintain the current behaviour
one must add keep frags to any ipfilter keep state rule (as documented
in the man pages).
This patch also allows the flexability to specify and use keep frags
separate from keep state, as documented in an example in ipf.conf.5,
instead of the currently broken behaviour.
Relnotes: yes
Memory is malloc'd, then a search for a match in the fragment table
is made and if the fragment matches, the wrong fragment table is
freed, causing a use after free panic. This commit fixes this.
A symptom of the problem is a kernel page fault in bcopy() called by
ipf_frag_lookup() at line 715 in ip_frag.c. Another symptom is a
kernel page fault in ipf_frag_delete() when called by ipf_frag_expire()
via ipf_slowtimer().
MFC after: 1 week
from the .depend files after the build:
cp -r ../vendor/edk2/MdePkg/Include sys/contrib/edk2
cd lib/libefivar
make
pushd `make -V .OBJDIR`
cat .depend*.o | grep sys/contrib | cut -d' ' -f 3 |
sort -u | sed -e 's=/full/path/sys/contrib/edk2/==' > /tmp/xxx
popd
cd ../../sys/contrib/edk2
rm -rf Include
for i in `cat /tmp/xxx`; do
svn cp svn+ssh://repo.freebsd.org/base/vendor/edk2/dist/MdePkg/$i $i
done
svn cp svn+ssh://repo.freebsd.org/base/vendor/edk2/dist/MdePkg/MdePkg.dec .
The original EDK2 repo is ~265MB, the MdePkg is ~23MB, all
MdePkg/Includes is ~7MB and this minimal set is ~1.3MB.
Sponsored by: Netflix
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
* flesh out a "get default DFS parameters" routine
* remove the stub that returns NULL
* fix up the enable DFS method to do what FreeBSD does - specifically, allow pe_enabled
to be set/cleared.
This allows the radar pulse reporting code to function, but it doesn't yet
do anything useful.
* add debugging
* disable the manual noise floor calibration and tracking done by the HAL;
this interferes with the normal calibration path and will lock up the RX
side
* don't program short report / priority if they're provided as NOVAL.
under a shared read lock. This patch attempts to upgrade the lock to
an exclusive write lock. If the exclusive write lock fails to be
obtained, the current fragment is not placed at the head of the list.
This portion of the patch was inspired by NetBSD ip_frag.c r1.4 (which
effectively removed the section of code that performed the reordering).
The patch to sys/contrib/ipfilter/netinet/ip_compat.h adds the
MUTEX_TRY_UPGRADE macro to support the patch to ip_frag.c.
The patch to contrib/ipfilter/lib/rwlock_emul.c supports this patch
by emulating the mutex in userspace when exercised by ipftest(1).
Inspired by: NetBSD ip_frag.c r1.4
MFC after: 1 month
Languages like C++17 and Go provide direct support for slice types:
pointer/length pairs. The CloudABI generator now has more complete for
this, meaning that for the C binding, pointer/length pairs now use an
automatic naming scheme of ${name} and ${name}_len.
Apart from this change and some reformatting, the ABI definitions are
identical. Binary compatibility is preserved entirely.
Include netinet/in.h before ip_compat.t which will then check if
IPPROTO_IPIP is defined or not. Doing it the other way round,
ip_compat.h would not find it defined and netinet/in.h then
redefine it.
Although vchiq_build_date and vchiq_build_time are not used in current
vchi driver at the moment, make sure these value will not leak into
build later on if at some point they will be refered in some new
imported code
PR: 215494
Reported by: emaste
MFC after: 1 week
Drop the tracking down to the pmap layer, with optimizations to only track
necessary pages. This should give a (slight) performance improvement, as well
as a stability improvement, as the tracking is already mostly handled by the
pmap layer.
As part of an effort to extend Book-E to the 64-bit world, make the necessary
changes to the DPAA/dTSEC driver set to be integer-pointer conversion clean.
This means no more casts to int, and use uintptr_t where needed.
Since the NCSW source is effectively obsolete, direct changes to the source tree
are safe.
All devices:
- add support for rate adaptation via ieee80211_amrr(9);
- use short preamble for transmitted frames when needed;
- multi-bss support:
* for RTL8821AU: 2 VAPs at the same time;
* other: 1 any VAP + 1 sta VAP.
RTL8188CE:
- fix IQ calibration bug (reason of significant speed degradation);
- add h/w crypto acceleration support.
USB:
- A-MPDU Tx support;
- short GI support;
Other:
- add support for RTL8812AU / RTL8821AU chipsets
(a/b/g/n only; no ac yet);
- split merged code into subparts:
* bus glue (usb/*, pci/*, rtl*/usb/*, rtl*/pci/*)
* common (if_rtwn*)
* chip-specific (rtl*/*)
- various other bugfixes.
Due to code reorganization, module names / requirements were changed too:
urtwn urtwnfw -> rtwn rtwn_usb rtwnfw
rtwn rtwnfw -> rtwn rtwn_pci rtwnfw
Tested with RTL8188CE, RTL8188CUS, RTL8188EU and RTL8821AU.
Tested by: kevlo, garga,
Peter Garshtja <peter.garshtja@ambient-md.com>,
Kevin McAleavey <kevin.mcaleavey@knosproject.com>,
Ilias-Dimitrios Vrachnis <id@vrachnis.com>,
<otacilio.neto@bsd.com.br>
Relnotes: yes
buildkernel failed with GCC 5.3 with
error: comparison of constant '852736' with boolean expression is always true
Sponsored by: The FreeBSD Foundation
Linux's copy of the Cavium SDK does not have these non-ASCII characters
and this reduces noise in diffs when comparing the two.
Sponsored by: The FreeBSD Foundation
Don't call pmap_kextract() multiple times, it wastes CPU cycles, which in a high
performance environment can be very expensive.
Inline XX_FindTracker() to allow more optimizations as well.
Since if_addrlist is used only for ipfilter(4), add a macro if_addrlist
in ip_compat.h.
Reviewed by: cy
Differential Revision: https://reviews.freebsd.org/D8059
alpine-hal SerDes file was omitted in the previous commit.
Files added here.
All unnecessary (old) files were also removed.
Merge from vendor-sys, r306017
- Use conditional instruction to simplify the ARMv6 vDSO. This means
that we no longer perform any branching. In the failure case, we
simply slide over the assignments of the return values.
The vDSO could be improved even further by using stmia to do
assignments in parallel. Unfortunately, the script used to generate
these is not smart enough for that (yet).
Spotted by: andrew@.
- Make the style of the i686 vDSO more similar to the others by using
decimal literals.
Bugs in the Python code used to generate this vDSO caused us to
miscompute the register numbers/stack offsets at which addresses of the
system call output arguments were stored.
Together with some other patches, this vDSO allows us to make all of the
cloudlibc unit tests pass.
Obtained from: https://github.com/NuxiNL/cloudabi
In order to make CloudABI work on ARMv6, start off by copying over the
sysvec for ARM64 and adjust it to use 32-bit registers. Also add code
for fetching arguments from the stack if needed, as there are fewer
register than on ARM64.
Also import the vDSO that is needed to invoke system calls. This vDSO
uses the intra procedure call register (ip) to store the system call
number. This is a bit simpler than what native FreeBSD does, as FreeBSD
uses r7, while preserving the original r7 into ip.
This sysvec seems to be complete enough to start CloudABI processes.
These processes are capable of linking in the vDSO and are therefore
capable of executing (most?) system calls successfully. Unfortunately,
the biggest show stopper is still that TLS is completely broken:
- The linker used by CloudABI, LLD, still has troubles with some of the
relocations needed for TLS. See LLVM bug 30218 for more details.
- Whereas FreeBSD uses the tpidruro register for TLS, for CloudABI I
want to make use of tpidrurw, so that userspace can modify the base
address directly. This is needed for efficient emulation.
Unfortunately, this register doesn't seem to be preserved across
context switches yet.
Obtained from: https://github.com/NuxiNL/cloudabi (the vDSO)
macro is defined in lots of different places in ipfilter, so replace all
of the nonportable definitions with portable ones.
Pointy hat to: dim
X-MFC-With: r304959, r304953
MFC after: 3 days
cnv API is a set of functions for managing name/value pairs by cookie.
The cookie can be obtained by nvlist_next(), nvlist_get_parent() or
nvlist_get_pararr() function. This patch also includes unit tests.
Submitted by: Adam Starak <starak.adam@gmail.com>
A nice thing about requiring a vDSO is that it makes it incredibly easy
to provide full support for running 32-bit processes on 64-bit systems.
Instead of letting the kernel be responsible for composing/decomposing
64-bit arguments across multiple registers/stack slots, all of this can
now be done in the vDSO. This means that there is no need to provide
duplicate copies of certain system calls, like the sys_lseek() and
freebsd32_lseek() we have for COMPAT_FREEBSD32.
This change imports a new vDSO from the CloudABI repository that has
automatically generated code in it that copies system call arguments
into a buffer, padding them to eight bytes and zero-extending any
pointers/size_t arguments. After returning from the kernel, it does the
inverse: extracting return values, in the process truncating
pointers/size_t values to 32 bits.
Obtained from: https://github.com/NuxiNL/cloudabi
The native CloudABI data types header file used to be pulled in by the
vDSOs when they were still written in C. Since they are now all
rewritten in assembly, this can go away.
Copy over amd64's cloudabi64_sysvec.c into i386 and tailor it to work.
Again, we use a system call convention similar to FreeBSD, except that
there is no support for indirect system calls (%eax == 0).
Where i386 differs from amd64 is that we have to store thread/process
entry arguments on the stack instead of using registers. We also have to
put an extra pointer on the stack for TLS (for GSBASE). Place that
pointer in the empty slot that is normally used to hold return
addresses. That seems to keep the code simple.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D7590
The reason why the old vDSOs were written in C using inline assembly was
purely because they were embedded in the C library directly as static
inline functions. This was practical during development, because it
meant you could invoke system calls without any library dependencies.
The vDSO was simply a copy of these functions.
Now that we require the use of the vDSO, there is no longer any need for
embedding them in C code directly. Rewriting them in assembly has the
advantage that they are closer to ideal (less useless branching, less
assumptions about registers remaining unclobbered by the kernel, etc).
They are also easier to build, as they no longer depend on the C type
information for CloudABI.
Obtained from: https://github.com/NuxiNL/cloudabi
Now that we've switched over to using the vDSO on CloudABI, it becomes a
lot easier for us to phase out old features. System call numbering is no
longer something that's part of the ABI. It's fully based on names. As
long as the numbering used by the kernel and the vDSO is consistent
(which it always is), it's all right.
Let's put this to the test by removing a system call (thread_tcb_set())
that's already unused for quite some time now, but was only left intact
to serve as a placeholder. Sync in the new system call table that uses
alphabetic sorting of system calls.
Obtained from: https://github.com/NuxiNL/cloudabi
Several files use the internal name of `struct device` instead of
`device_t` which is part of the public API. This patch changes all
`struct device *` to `device_t`.
The remaining occurrences of `struct device` are those referring to the
Linux or OpenBSD version of the structure, or the code is not built on
FreeBSD and it's unclear what to do.
Submitted by: Matthew Macy <mmacy@nextbsd.org> (previous version)
Approved by: emaste, jhibbits, sbruno
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D7447
CloudABI executables that are emulated on Mac OS X do not invoke system
calls through "syscall". Instead, they make use of a vDSO that is
provided by the emulator that provides symbols for all of the system
call routines. The emulator can implement these any way it likes.
At some point in time we want to do this for native execution as well,
so that CloudABI executables are entirely oblivious of how system calls
need to be performed. They will simply call into functions and let that
deal with all of the details.
These source files can be used to generate a simple vDSO that does
nothing more than invoke "syscall". All we need to do now is map it into
the processes.
Obtained from: https://github.com/NuxiNL/cloudabi
f/w for the other devices supported by this driver.
Patch linked in https://reviews.freebsd.org/D6967 but not actually
a part of the review.
Obtained from DragonflyBSD.
Submitted by: Kevin Bowling <kev009@kev009.com>
MFC after: 2 weeks
Relnotes: yes
