Commit Graph

803 Commits

Author SHA1 Message Date
Hans Petter Selasky
01fdacdbc7 Allow the list header file in the LinuxKPI to be used in standalone code.
Some style and spelling nits while at it.

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-12-04 15:46:48 +00:00
Hans Petter Selasky
ed2b70e8af Use function macro for sema_init() in the LinuxKPI to limit macro expansion scope.
MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-30 09:47:53 +00:00
Hans Petter Selasky
99f20bdc47 Allow LinuxKPI types to be used in bootloaders, by checking for the
_STANDALONE definition.

No functional change intended.

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-18 13:47:11 +00:00
Vladimir Kondratyev
146e176df7 LinuxKPI: Exclude linux/acpi.h content on non-ACPI archs.
LinuxKPI ACPI support is based on FreeBSD import of ACPICA which can be
compiled only on aarch64, amd64 and i386. Ifdef-out broken parts on our
side to avoid patching of vendor code.

This fixes drm-devel-kmod build on powerpc64(le).

Reported by:	pkubaj
2020-11-14 10:34:18 +00:00
Emmanuel Vadot
dab39c11af LinuxKPI: Implement ACPI bits required by drm-kmod in base system
It includes:

ACPI_HANDLE() implementation.
AC and VIDEO ACPI events notification support.
Replacement of hand-rolled GPLed _DSM method evaluation helpers
with in-base ones.

Submitted by:	wulf
Differential Revision:	https://reviews.freebsd.org/D26603
2020-11-09 13:20:14 +00:00
Mateusz Guzik
e90afaa015 kqueue: save space by using only one func pointer for assertions 2020-11-09 00:04:35 +00:00
Hans Petter Selasky
ab79c9061c Implement xa_init() in the LinuxKPI as a wrapper for xa_init_flags().
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-24 13:16:10 +00:00
Hans Petter Selasky
e23ee5b884 Remove ifdefs around IS_ALIGNED() definition in the LinuxKPI.
Discussed with:		manu@
MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-16 11:01:21 +00:00
Hans Petter Selasky
d524c46fb8 Implement more RCU list functions in the LinuxKPI.
This also fixes a bug in the existing list_add_rcu() where the
prev->prev pointer was updated to the new element instead of
next->prev. Currently this function is not widely used.

MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-13 16:19:21 +00:00
Bjoern A. Zeeb
90707c4e44 LinuxKPI: add a bitfield.h implementation.
This code was iteratively implemented during the work on various WiFi
drivers -- from individual functions to a macro-created implementations
for the various bit sized needed (and then extended to more for
comepleteness). Some of the bit combinations do not seem to make sense
so are left out.

The __bf_shf(x) was obtained from D26681 [1].

Requested by:		manu [1]
Reviewed by:		hselasky, manu
MFC after:		1 week
Sponsored by:		The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26708
2020-10-07 22:07:26 +00:00
Emmanuel Vadot
a113b1037f linuxkpi: Add pagemap.h
Add release_pages needed by drm which simply calls put_page for
all the pages provided

Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26680
2020-10-06 10:41:00 +00:00
Emmanuel Vadot
b74986e7fa linuxkpi: Add power_supply.h
Add power_supply_is_system_supplied which is needed by drm.

Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26679
2020-10-06 10:39:40 +00:00
Emmanuel Vadot
49c85a33e5 linuxkpi: Add prefetch.h
Only add prefetchw as it is the only function used by drm.
Simply use the __builtin_prefetch which is available in all
compiler for a long time.

Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26677
2020-10-06 10:37:21 +00:00
Emmanuel Vadot
3ee75811a6 linuxkpi: Add numa.h
Only contain NUMA_NO_NODE needed by drm

Reviewed by:	bz
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26676
2020-10-06 10:36:16 +00:00
Emmanuel Vadot
2aa0ea94ea linuxkpi: Add gcd function
This compute the common greater divider
Taken from OpenBSD

Reviewed by:	bz, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26674
2020-10-06 10:35:03 +00:00
Hans Petter Selasky
4c2dddd8a7 Populate the acquire context field of a ww_mutex in the LinuxKPI.
Bump the FreeBSD version to force recompilation of external kernel modules.

MFC after:		1 week
Differential Revision:	https://reviews.freebsd.org/D26657
Submitted by:		greg_unrelenting.technology (Greg V)
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-04 17:23:39 +00:00
Emmanuel Vadot
a91b408a36 linuxkpi: Add dmi_* function
dmi function are used to get smbios values.
The DRM subsystem and drivers use it to enabled (or not) quirks.

Reviewed by:	hselasky
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26046
2020-10-02 18:28:00 +00:00
Emmanuel Vadot
2b68c97337 linuxkpi: Add backlight support
Add backlight function to linuxkpi.
Graphics drivers expose the backlight of the panel directly so allow them to use the backlight subsystem so
user can use backlight(8) to configure them.

Reviewed by:	hselasky
Relnotes:	yes
Differential Revision:	The FreeBSD Foundation
2020-10-02 18:26:41 +00:00
Mateusz Guzik
1a18003240 compat: clean up empty lines in .c and .h files 2020-09-01 21:24:33 +00:00
Vladimir Kondratyev
5d4bf0578f LinuxKPI: Implement ksize() function.
In Linux, ksize() gets the actual amount of memory allocated for a given
object. This commit adds malloc_usable_size() to FreeBSD KPI which does
the same. It also maps LinuxKPI ksize() to newly created function.

ksize() function is used by drm-kmod.

Reviewed by:	hselasky, kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D26215
2020-08-29 19:26:31 +00:00
Hans Petter Selasky
d96e599643 Implement extensible arrays API using the existing radix tree implementation
in the LinuxKPI.

Differential Revision:	https://reviews.freebsd.org/D25101
Reviewed by:	kib @
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-08-27 10:28:12 +00:00
Mateusz Guzik
a92a971bbb vfs: remove the thread argument from vget
It was already asserted to be curthread.

Semantic patch:

@@

expression arg1, arg2, arg3;

@@

- vget(arg1, arg2, arg3)
+ vget(arg1, arg2)
2020-08-16 17:18:54 +00:00
Emmanuel Vadot
0e123c13fe linuxkpi: Add a few wait_bit functions
The linux function does a lot more than that as multiple waitqueue could be fetch
from a static table based on the hash of the argument but since in DRM it's only used
in one place just add a single variable.
We will probably need to change that in the futur but it's ok with DRM even with current
linux.

Reviewed by:	hselasky
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26054
2020-08-14 08:48:17 +00:00
Hans Petter Selasky
74d3a63559 Use atomic_clear_rel_long() to implement clear_bit_unlock() in the LinuxKPI
after r363842.

Suggested by:	alc@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-08-11 12:41:40 +00:00
Hans Petter Selasky
6ae240797f Need to clone the task struct fields related to RCU aswell in the
LinuxKPI after r359727. This fixes a minor regression issue. Else the
priority tracking won't work properly when both sleepable and
non-sleepable RCU is in use on the same thread.

Bump the __FreeBSD_version to force recompilation of external kernel
modules.

PR:		242272
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-08-11 12:17:46 +00:00
Mateusz Guzik
51ea7bea91 vfs: add VOP_STAT
The current scheme of calling VOP_GETATTR adds avoidable overhead.

An example with tmpfs doing fstat (ops/s):
before: 7488958
after:  7913833

Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D25910
2020-08-07 23:06:40 +00:00
Hans Petter Selasky
6b839ff47b Implement radix_tree_store() in the LinuxKPI for use with the coming
extensible arrays implementation.

While at it add some more comments explaining the current
radix_tree_insert() function and make sure to clean the root node when
the radix tree reaches the maximum height. This can happen if the
index passed is too big when the tree is empty.

The radix_tree_store() function is basically a copy of the
radix_tree_insert() function with some added functionality.

The radix_tree_store() function is local to FreeBSD and does not yet
exist in Linux.

Reviewed by:		kib
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-08-07 16:15:44 +00:00
Emmanuel Vadot
dfb4ecb38b linuxkpi: Add time_after32 and time_before32
This compare two 32 bits times

Sponsored by: The FreeBSD Foundation
Reviewed by:	kib, hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25700
2020-08-04 15:27:32 +00:00
Emmanuel Vadot
334680ab07 linuxkpi: Add clear_bit_unlock
This calls clear_bit and adds a memory barrier.

Sponsored by: The FreeBSD Foundation

Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25943
2020-08-04 15:25:22 +00:00
Emmanuel Vadot
38ba9c8bac Re-apply r363564.
We now have linux/sizes.h in the tree.
2020-08-04 14:53:41 +00:00
Emmanuel Vadot
2d946b2e12 linuxkpi: Add nested variant of mutex_lock_interruptible
We don't do anything with the _nesteds variant so just call mutex_lock_interruptible

Sponsoredby: The FreeBSD Foundation
Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25944
2020-08-04 14:45:22 +00:00
Emmanuel Vadot
7237a74f3b linuxkpi: Add kref_put_lock
Same as kref_put but in addition to calling the rel function it will
acquire the lock first.

Sponsored by: The FreeBSD Foundation
Reviewed by:	hselasky, emaste
Differential Revision:	https://reviews.freebsd.org/D25942
2020-08-04 14:44:16 +00:00
Emmanuel Vadot
16fdd8b7ad linuxkpi: Add linux/sizes.h
This file contain some defines for common sizes.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky, emaste
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25941
2020-08-04 14:42:38 +00:00
Emmanuel Vadot
85d787b2fe Fix r363565
lockdep.h needs sys/lock.h for LOCK_CLASS
2020-07-26 18:33:29 +00:00
Emmanuel Vadot
cdb6eebe08 Revert r363564
linux/sizes.h doesn't exists in base ... sorry.
2020-07-26 17:21:24 +00:00
Emmanuel Vadot
0e4e9e8f34 linuxkpi: Add taint* defines
This isn't used for us but allow us to port drivers more easily.

Reviewed by:	hselasky
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25703
2020-07-26 16:31:49 +00:00
Emmanuel Vadot
f12af2b387 linuxkpi: Include hardirq.h in preempt.h and lockdep.h in hardirq.h
Linux does the same, this avoids ifdef or extra includes in ported drivers.

Reviewed by:	emaste, hselasky
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25702
2020-07-26 16:30:59 +00:00
Emmanuel Vadot
820272c408 linuxkpi: Include linux/sizes.h in dma-mapping.h
Linux does the same, this avoids ifdef or extra includes in ported drivers.

Reviewed by:	emaste, hselasky
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D25701
2020-07-26 16:30:01 +00:00
Mark Johnston
94140f4781 usb(4): Stop checking for failures from malloc(M_WAITOK).
Handle the fact that parts of usb(4) can be compiled into the boot
loader, where M_WAITOK does not guarantee a successful allocation.

PR:		240545
Submitted by:	Andrew Reiter <arr@watson.org> (original version)
Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25706
2020-07-22 14:32:47 +00:00
Vladimir Kondratyev
34c2f79d83 linuxkpi: Ignore NULL pointers passed to string parameter of kstr(n)dup
That follows Linux and fixes related drm-kmod-5.3 panic.

Reviewed by:	imp, hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25657
2020-07-14 21:56:59 +00:00
Hans Petter Selasky
127d8cfafb Implement the bitmap_subset() function in the LinuxKPI. This function
checks if the bitmap pointed to by the first argument is a subset of
the bitmap pointed to by the second argument. The function returns one
on success and zero on failure.

MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-07-10 12:06:18 +00:00
Hans Petter Selasky
d2890eeea1 Implement the array_size() function in the LinuxKPI. This function
basically multiplies its two arguments and returns SIZE_MAX if the
result overflows the size_t type.  Else the product of the two
arguments is returned.

Bump the FreeBSD_version to mitigate issues with existing
implementation of array_size() in drm-devel-kmod.

Discussed with:		manu@
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-07-10 11:27:54 +00:00
Hans Petter Selasky
588fbadffb Fix include file order in io.h in the LinuxKPI.
Make sure sys/types.h is included before machine/vm.h.

PR:		247775
Submitted by:	pkubaj@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-07-05 19:38:36 +00:00
Konstantin Belousov
f334f212d9 linuxkpi: improvements for linux_pid_task() and linux_get_pid_task().
Unify functions bodies.
Do not call tdfind() if pid is passed, and do not call pfind() if tid
is supplied.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D25534
2020-07-02 10:42:58 +00:00
Hans Petter Selasky
9a4e535b39 The "pid" field in the LinuxKPI task struct is typically set to the thread ID
and not the process ID. Make sure the linux_task_exiting() function uses tdfind()
to lookup the BSD procedure structure pointer by the "pid" field, and only
fallback to pfind() when no match is found! This makes linux_task_exiting()
in line with the rest of the code.

Differential Revision: https://reviews.freebsd.org/D25509
Submitted by:	Greg V <greg@unrelenting.technology>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-07-01 08:23:57 +00:00
Hans Petter Selasky
d326a6c7c1 Document the is_signed(), type_max() and type_min() function macros in the
LinuxKPI. Try to make the function argument more readable.

Suggested by:	several
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-30 08:41:33 +00:00
Hans Petter Selasky
d0eed838e3 Implement is_signed(), type_max() and type_min() function macros in the
LinuxKPI.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-29 13:08:40 +00:00
Mark Johnston
3507b8d467 Remove some redundant assignments and computations.
Reported by:	alc
Reviewed by:	alc, kib
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25400
2020-06-28 21:34:38 +00:00
Doug Moore
158c55a584 In r362552, RB_SET_PARENT is defined, and use in parens in
RB_CLEAR_NODE.  But it is not an expression, and ought not to be
enclosed in parens.  Remove them.

Approved by:	markj
Differential Revision:	https://reviews.freebsd.org/D25421
2020-06-23 22:47:54 +00:00
Doug Moore
4d56980017 Define RB_SET_PARENT to do all assignments to rb parent
pointers. Define RB_SWAP_CHILD to replace the child of a parent with
its twin, and use it in 4 places. Use RB_SET in rb_link_node to remove
the only linuxkpi reference to color, and then drop color- and
parent-related definitions that are defined and used only in rbtree.h.

This is intended to be entirely cosmetic, with no impact on program
behavior, and leave RB_PARENT and RB_SET_PARENT as the only ways to
read and write rb parent pointers.

Reviewed by:	markj, kib
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D25264
2020-06-23 20:02:55 +00:00
Mark Johnston
0f1e6ec591 Add a helper function for validating VA ranges.
Functions which take untrusted user ranges must validate against the
bounds of the map, and also check for wraparound.  Instead of having the
same logic duplicated in a number of places, add a function to check.

Reviewed by:	dougm, kib
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25328
2020-06-19 03:32:04 +00:00
Doug Moore
9f1041dc2e Linuxkpi uses the rb-tree structures without using their interfaces,
making them break when the representation changes. Revert changes that
eliminated the color field from rb-trees, leaving everything as it was
before.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D25250
2020-06-13 01:54:09 +00:00
Doug Moore
13dca1937f Revert r362108, as it breaks compilation. 2020-06-12 17:48:12 +00:00
Doug Moore
3159ceca97 The linuxkpi code accesses left/right rb tree pointers without using
RB_LEFT or RB_RIGHT, so they aren't stripping off the color bit
encoded there. Strip off that bit for linuxkpi.

Reported by:	dch
Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D25245
2020-06-12 16:51:55 +00:00
Doug Moore
36ba4b393f To reduce the size of an rb_node, drop the color field. Set the least
significant bit in the pointer to the node from its parent to indicate
that the node is red. Have the tree rotation macros leave the
old-parent/new-child node red and the new-parent/old-child node black.

This change makes RB_LEFT and RB_RIGHT no longer assignable, and
RB_COLOR no longer defined. Any code that modifies the tree or
examines a node color would have to be modified after this change.

Reviewed by:	markj
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D25105
2020-06-09 20:19:11 +00:00
Hans Petter Selasky
c51613866f Ensure pci_channel_offline() actually queries the PCI register space,
and not only the software cache of that register.  Else
pci_channel_offline() won't detect that the PCI device is gone when
using the LinuxKPI.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-05 08:12:08 +00:00
Hans Petter Selasky
d053391cd7 Implement __is_constexpr() function macro in the LinuxKPI.
Bump the FreeBSD version.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-02 12:23:04 +00:00
Hans Petter Selasky
ef5f8c18b5 Implement struct_size() function macro in the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-02 10:19:45 +00:00
Hans Petter Selasky
c185f13b92 Implement BUILD_BUG_ON_ZERO() in the LinuxKPI.
Tested using gcc and clang.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-06-02 09:45:43 +00:00
Emmanuel Vadot
8e52f22b25 linuxkpi: Add kstrtou16
This function convert a char * to a u16.
Simply use strtoul and cast to compare for ERANGE

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24996
2020-05-27 11:42:09 +00:00
Emmanuel Vadot
9b703f3082 linuxkpi: Add rcu_swap_protected
This macros swap an rcu pointer with a normal pointer.
The condition only seems to be used for debug/warning under linux, ignore
for now.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24954
2020-05-27 10:01:30 +00:00
Emmanuel Vadot
8287045dde linuxkpi: Add overflow.h
Only add check_add_overflow and check_mul_overflow as those are the only
two needed function by DRM v5.3.
Both gcc and clang have builtin to do this check so use them directly
but throw an error if the compiler/code checker doesn't support this builtin.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselsasky
Differential Revision:	https://reviews.freebsd.org/D25015
2020-05-27 09:31:50 +00:00
Emmanuel Vadot
42f0f394f7 linuxkpi: Fix mod_timer and del_timer_sync
mod_timer is supposed to return 1 if the modified timer was pending, which
is exactly what callout_reset does so return the value after checking
that it's a correct one in case the api change.
del_timer_sync returns int so add a function and handle that.

Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24983
2020-05-25 12:46:05 +00:00
Emmanuel Vadot
4efd5dd70d linuxkpi: Add refcount.h
Implement some refcount functions needed by drm.
Just use the atomic_t struct and functions from linuxkpi for simplicity.

Sponsored-by: The FreeBSD Foundation

Reviewed by:	hselsasky
Differential Revision:	https://reviews.freebsd.org/D24985
2020-05-25 12:44:07 +00:00
Emmanuel Vadot
93d70cd3fe linuxkpi: Add __same_type and __must_be_array macros
The same_type macro simply wraps around builtin_types_compatible_p which
exist for both GCC and CLANG, which returns 1 if both types are the same.
The __must_be_array macros returns 1 if the argument is an array.

This is needed for DRM v5.3

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24953
2020-05-25 12:42:55 +00:00
Emmanuel Vadot
af300929f9 linuxkpi: Add prandom_u32_max
This is just a wrapper around arc4random_uniform
Needed by DRM v5.3

Sponsored-by: The FreeBSD Foundation
Reviewed by:	cem, hselasky
Differential Revision:	https://reviews.freebsd.org/D24961
2020-05-23 17:52:25 +00:00
Emmanuel Vadot
2491b25c3a linuxkpi: Add rcu_work functions
The rcu_work function helps to queue some work after waiting for a grace
period.
This is needed by DRM drivers.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24942
2020-05-21 20:18:38 +00:00
Emmanuel Vadot
eda697d2ef linuxkpi: Add irq_work.h
Since handlers are call in a thread context we can simply use a workqueue
to emulate those functions.
The DRM code was patched to do that already, having it in linuxkpi allows us
to not patch the upstream code.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24859
2020-05-19 09:04:35 +00:00
Emmanuel Vadot
5e30a739c7 linuxkpi: add pci_dev_present
pci_dev_present shows if a set of pci ids are present in the system.
It just wraps pci_find_device.
Needed by DRMv5.2

Submitted by:	Austing Shafer (ashafer@badland.io)
Differential Revision:	https://reviews.freebsd.org/D24796
2020-05-19 08:44:33 +00:00
Emmanuel Vadot
ff443195bf linuxkpi: Add __init_waitqueue_head
The only difference with init_waitqueue_head is that the name and the
lock class key are provided but we don't use those so use init_waitqueue_head
directly.

Sponsored-by: The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24861
2020-05-19 08:43:17 +00:00
Emmanuel Vadot
355711ea76 linuxkpi: Add offsetofend macro
This calculate the offset of the end of the member in the given struct.
Needed by DRM in Linux v5.3

Sponsored-by: The FreeBSD Foudation
Differential Revision:	https://reviews.freebsd.org/D24849
2020-05-17 20:14:49 +00:00
Emmanuel Vadot
d003cc4318 linuxkpi: Add __mutex_init
Same as mutex_init, the lock_class_key argument seems to be only used for
debug in Linux, simply ignore it for now.
Needed by DRM in Linux v5.3

Sponsored-by: The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24848
2020-05-17 20:12:16 +00:00
Emmanuel Vadot
7708d3d765 linuxkpi: Add atomic_dec_and_mutex_lock
This function decrement the counter and if the result is 0 it acquires
the mutex and returns 1, if not it simply returns 0.
Needed by DRM from Linux v5.3

Sponsored-by: The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24847
2020-05-17 20:09:11 +00:00
Hans Petter Selasky
cf9f2ca3ef Implement synchronize_srcu_expedited() in the LinuxKPI.
Differential Revision:	https://reviews.freebsd.org/D24798
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-16 14:27:50 +00:00
Emmanuel Vadot
cfa985350d linuxkpi: Add EBADRQC to errno.h
This is used in the amdgpu driver from Linux 5.2

Sponsored-by: The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D24807
2020-05-13 07:49:12 +00:00
Andriy Gapon
a164a32b4d linuxkpi: print stack trace in WARN_ON macros
Reviewed by:	hselasky, kib
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D24779
2020-05-13 07:47:56 +00:00
Emmanuel Vadot
3d84874da0 linuxkpi: Really add bitmap_alloc and bitmap_zalloc
This was missing in r360870

Sponsored-by: The FreeBSD Foundation
2020-05-10 13:12:05 +00:00
Emmanuel Vadot
ce03b3013f linuxkpi: Add bitmap_alloc and bitmap_free
This is a simple call to kmallock_array/kfree, therefore include linux/slab.h as
this is where the kmalloc_array/kfree definition is.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselsasky
Differential Revision:	https://reviews.freebsd.org/D24794
2020-05-10 13:07:00 +00:00
Emmanuel Vadot
26a578697c linuxkpi: Add bitmap_copy and bitmap_andnot
bitmap_copy simply copy the bitmaps, no idea why it exists.
bitmap_andnot is similar to bitmap_and but uses !src2.

Sponsored-by: The FreeBSD Foundation
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D24782
2020-05-09 17:52:50 +00:00
Emmanuel Vadot
4c27484934 linuxkpi: Add pci_iomap and pci_iounmap
Those function are use to map/unmap io region of a pci device.
Different resource can be mapped depending on the bar so use a
tailq to store them all.

Sponsored-by: The FreeBSD Foundation

Reviewed by:	emaste, hselasky
Differential Revision:	https://reviews.freebsd.org/D24696
2020-05-07 17:00:51 +00:00
Hans Petter Selasky
5e6233ccab Optimise use of sg_page_count() in __sg_page_iter_next() in the LinuxKPI.
No need to compute value twice.

No functional change intended.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-04 10:10:07 +00:00
Hans Petter Selasky
fe4b041a14 Implement more scatter and gather functions in the LinuxKPI.
Differential Revision:	https://reviews.freebsd.org/D24611
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-04 09:58:45 +00:00
Hans Petter Selasky
42f8ef4bf5 Fix warning about sleeping with non-sleepable lock when allocating
"current" from linux_cdev_pager_populate() in the LinuxKPI:

Backtrace:
witness_debugger()
witness_warn()
uma_zalloc_arg()
malloc()
linux_alloc_current()
linux_cdev_pager_populate()
vm_fault()
vm_fault_trap()
trap_pfault()
trap()
calltrap()

Suggested by:	avg@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-04 08:05:01 +00:00
Hans Petter Selasky
b4edb17c82 Implement more PCI-express bandwidth functions in the LinuxKPI.
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-01 10:32:42 +00:00
Hans Petter Selasky
1bbbe083a1 Implement mutex_lock_killable() in the LinuxKPI.
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-01 10:28:21 +00:00
Hans Petter Selasky
3ff7ec1cc1 Implement DIV64_U64_ROUND_UP() in the LinuxKPI.
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-01 10:25:07 +00:00
Hans Petter Selasky
922106bf00 Implement more lockdep macros in the LinuxKPI.
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-01 10:18:07 +00:00
Hans Petter Selasky
61f7fe6b2d Implement kstrtou64() in the LinuxKPI.
Submitted by:	ashafer_badland.io (Austin Shafer)
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-01 10:14:45 +00:00
Hans Petter Selasky
253dbe7487 Factor code in LinuxKPI to allow attach and detach using any BSD device.
This allows non-LinuxKPI based infiniband device drivers to attach
correctly to ibcore.

No functional change intended.

Reviewed by:	np @
Differential Revision:	https://reviews.freebsd.org/D24514
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-22 14:33:25 +00:00
Hans Petter Selasky
b9bf16adfb Implement the atomic fetch add unless functions for the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-20 16:21:37 +00:00
Hans Petter Selasky
fdbfa4f19e Implement aligned LinuxKPI types for u16, u32 and u64.
Makes a difference for 32-bit platforms mostly.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-20 14:03:05 +00:00
Hans Petter Selasky
07fdea3672 Allow test_bit() in the LinuxKPI to accept a const pointer.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-20 13:47:15 +00:00
Hans Petter Selasky
47c0672b08 Allow the ERR_CAST() function in the LinuxKPI to take a const void pointer.
No functional change.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-20 13:36:01 +00:00
Mark Johnston
546c117f86 Remove a vestigal reference to kmem_object.
kmem_object has been an alias of kernel_object for a while.

MFC after:	1 week
2020-04-17 19:12:52 +00:00
Hans Petter Selasky
eae5868ce9 Clone the RCU interface into a sleepable and a non-sleepable part
in the LinuxKPI.

This allows synchronize RCU to be used inside a SRCU read section.
No functional change intended.

Bump the __FreeBSD_version to force recompilation of external kernel modules.

PR:		242272
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-08 17:09:45 +00:00
Hans Petter Selasky
61d82b0794 Some fixes for SRCU in the LinuxKPI.
- Make sure to use READ_ONCE() when deferring variables.
- Remove superfluous zero initializer.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-04-08 16:07:57 +00:00
Warner Losh
9275cd0dc5 Implement a workaround for kms-drm modules
pci_iov_if.h was added to pci.h, but none of the kms-drm branches have
that. Rather than play whack a mole with the branches, move its inclusion to
linux_pci.c which is the only part of the code that needs it now.

Longer term, other solutions will be needed, but this gives us time to get those
deployed on all the supported versions.
2020-03-20 15:07:25 +00:00
Konstantin Belousov
2928e60e55 linuxkpi: Add infrastructure to pass FreeBSD IOV method calls into
pci_driver methods.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:10:49 +00:00
Hans Petter Selasky
1328771d9d When closing a LinuxKPI file always use the real release function to avoid
resource leakage when destroying a LinuxKPI character device.

Submitted by:	Andrew Boyer <aboyer@pensando.io>
Reviewed by:	kib@
PR:		244572
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-03-03 15:49:34 +00:00
Hans Petter Selasky
77632fc70f Extend the range of the return value from nsecs_to_jiffies64() to support
Mesa's drm_syncobj usage, in the LinuxKPI.

While at it optimise the jiffies conversion functions to avoid repeated
and constant calculations.

Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D23846
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-02-27 15:21:05 +00:00
Emmanuel Vadot
1179b649cf linuxkpi: Move shmem related functions in it's own file
For drmkpi (D23085) we don't want the Linux struct file as we don't emulate
everything. Also the prototypes should be in shmem_fs.h to have 100%
compatibility with Linux.

Reviewed by:	hselasky
MFC after:	Maybe
Differential Revision:	https://reviews.freebsd.org/D23764
2020-02-21 09:28:45 +00:00
Emmanuel Vadot
1a7ba9a01c linuxkpi: Add str_has_prefix
This function test if the string str begins with the string pointed
at by prefix.

Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D23767
2020-02-20 17:20:50 +00:00
Emmanuel Vadot
8f0c734385 linuxkpi: Add list_is_first function
This function just test if the element is the first of the list.

Reviewed by:	hselasky
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D23766
2020-02-20 17:19:16 +00:00
Pawel Biernacki
39a3542bef Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (2 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked). Use it in
preparation for a general review of all nodes.
This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Reviewed by:	hselasky, kib, zeising
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D23631
2020-02-15 18:54:59 +00:00
Mateusz Guzik
52604ed792 fd: remove the seq argument from fget_unlocked
It is almost always NULL.
2020-02-03 22:27:55 +00:00
Konstantin Belousov
2a3529df1d Provide support for fdevname(3) on linuxkpi-backed devices.
Reported and tested by:	manu
Reviewed by:	hselasky, manu
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D23386
2020-01-28 11:22:20 +00:00
Hans Petter Selasky
b9a6330da3 Implement mmget_not_zero() in the LinuxKPI.
Submitted by:	Austin Shafer <ashafer@badland.io>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-01-24 13:05:53 +00:00
Mark Johnston
a7e348d7cf Handle a NULL thread pointer in linux_close_file().
This can happen if a file is closed during unix socket GC.  The same bug
was fixed for devfs descriptors in r228361.

PR:		242913
Reported and tested by:	iz-rpi03@hs-karlsruhe.de
Reviewed by:	hselasky, kib
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23178
2020-01-15 15:31:35 +00:00
Konstantin Belousov
fedab1b499 Code must not unlock a mutex while owning the thread lock.
Reviewed by:	hselasky, markj
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D23150
2020-01-13 14:30:19 +00:00
Mateusz Guzik
b249ce48ea vfs: drop the mostly unused flags argument from VOP_UNLOCK
Filesystems which want to use it in limited capacity can employ the
VOP_UNLOCK_FLAGS macro.

Reviewed by:	kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D21427
2020-01-03 22:29:58 +00:00
Mark Johnston
aa2ad8c1e6 Remove set_page_dirty_lock().
Its use of the page lock is incorrect, and it is not used by the DRM
modules.

Reviewed by:	hselasky
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D23002
2020-01-02 19:29:14 +00:00
Jeff Roberson
3cf3b4e641 Make page busy state deterministic on free. Pages must be xbusy when
removed from objects including calls to free.  Pages must not be xbusy
when freed and not on an object.  Strengthen assertions to match these
expectations.  In practice very little code had to change busy handling
to meet these rules but we can now make stronger guarantees to busy
holders and avoid conditionally dropping busy in free.

Refine vm_page_remove() and vm_page_replace() semantics now that we have
stronger guarantees about busy state.  This removes redundant and
potentially problematic code that has proliferated.

Discussed with:	markj
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D22822
2019-12-22 06:56:44 +00:00
Hans Petter Selasky
cc79ea3a26 Restore important comment in RCU/EPOCH support in FreeBSD after r355784.
Sponsored by:	Mellanox Technologies
2019-12-18 09:30:32 +00:00
Jeff Roberson
686bcb5c14 schedlock 4/4
Don't hold the scheduler lock while doing context switches.  Instead we
unlock after selecting the new thread and switch within a spinlock
section leaving interrupts and preemption disabled to prevent local
concurrency.  This means that mi_switch() is entered with the thread
locked but returns without.  This dramatically simplifies scheduler
locking because we will not hold the schedlock while spinning on
blocked lock in switch.

This change has not been made to 4BSD but in principle it would be
more straightforward.

Discussed with:	markj
Reviewed by:	kib
Tested by:	pho
Differential Revision: https://reviews.freebsd.org/D22778
2019-12-15 21:26:50 +00:00
Jeff Roberson
61a74c5ccd schedlock 1/4
Eliminate recursion from most thread_lock consumers.  Return from
sched_add() without the thread_lock held.  This eliminates unnecessary
atomics and lock word loads as well as reducing the hold time for
scheduler locks.  This will eventually allow for lockless remote adds.

Discussed with:	kib
Reviewed by:	jhb
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D22626
2019-12-15 21:11:15 +00:00
Conrad Meyer
482f0c0255 Revert r355760, r355759
And remove the inline/deprecated attribute use entirely in stdlib.h, from
r355747.  The intent was to provide a buildable API transitionary period, but
clearly that was counter-productive.

Reported by:	delphij, imp, others
2019-12-15 17:33:26 +00:00
Conrad Meyer
9d4710591b linuxkpi: Drop incompatible __deprecated definition
Probably all of these linuxkpi stubs should be '#ifndef' guarded, but maybe
that would prevent people from noticing when they are defined.

Introduced in r355759.  For some reason I only ran a buildworld and not a
kernel.  Mea culpa.

Reported by:	Mark Millard
X-MFC-with:	r355759
2019-12-14 23:39:32 +00:00
Hans Petter Selasky
76354fa40f Enable device class group attributes in the LinuxKPI.
Bump the __FreeBSD_version to force recompilation of
external kernel modules due to structure change.

Differential Revision:	https://reviews.freebsd.org/D21564
Submitted by:	Greg V <greg@unrelenting.technology>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-11-04 14:19:09 +00:00
Ryan Stone
92a15f946b Add missing M_NOWAIT flag
The LinuxKPI linux_dma code calls PCTRIE_INSERT with a
mutex held, but does not set M_NOWAIT when allocating
nodes, leading to a potential panic.  All of this code
can handle an allocation failure here, so prefer an
allocation failure to sleeping on memory.

Also fix a related case where NOWAIT/WAITOK was not
specified.  In this case it's not clear whether sleeping
is allowed so be conservative and assume not.  There are
a lot of other paths in this code that can fail due to
a lack of memory anyway.

Differential Revision: https://reviews.freebsd.org/D22127
Reviewed by: imp
Sponsored by: Dell EMC Isilon
MFC After: 1 week
2019-10-23 17:20:20 +00:00
Hans Petter Selasky
427c402de2 Fix missing epochification of the LinuxKPI after r353292.
Sponsored by:	Mellanox Technologies
2019-10-15 11:14:14 +00:00
Jeff Roberson
0012f373e4 (4/6) Protect page valid with the busy lock.
Atomics are used for page busy and valid state when the shared busy is
held.  The details of the locking protocol and valid and dirty
synchronization are in the updated vm_page.h comments.

Reviewed by:    kib, markj
Tested by:      pho
Sponsored by:   Netflix, Intel
Differential Revision:        https://reviews.freebsd.org/D21594
2019-10-15 03:45:41 +00:00
Hans Petter Selasky
4c8ba7d94f Use true and false when dealing with bool type in the LinuxKPI.
No functional change.

MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-09-11 08:24:47 +00:00
Hans Petter Selasky
16732c193c Fix synchronous work drain issue in the LinuxKPI.
A work callback may restart itself. Loop in the drain function to see if the
work has been rescheduled and stop the subsequent reschedules, if any.

MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-09-11 08:20:13 +00:00
Hans Petter Selasky
6575da5eef Fix broken DECLARE_TASKLET() macro after r347852.
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-09-11 07:53:49 +00:00
Jeff Roberson
c75757481f Replace redundant code with a few new vm_page_grab facilities:
- VM_ALLOC_NOCREAT will grab without creating a page.
 - vm_page_grab_valid() will grab and page in if necessary.
 - vm_page_busy_acquire() automates some busy acquire loops.

Discussed with:	alc, kib, markj
Tested by:	pho (part of larger branch)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D21546
2019-09-10 19:08:01 +00:00
Mark Johnston
fee2a2fa39 Change synchonization rules for vm_page reference counting.
There are several mechanisms by which a vm_page reference is held,
preventing the page from being freed back to the page allocator.  In
particular, holding the page's object lock is sufficient to prevent the
page from being freed; holding the busy lock or a wiring is sufficent as
well.  These references are protected by the page lock, which must
therefore be acquired for many per-page operations.  This results in
false sharing since the page locks are external to the vm_page
structures themselves and each lock protects multiple structures.

Transition to using an atomically updated per-page reference counter.
The object's reference is counted using a flag bit in the counter.  A
second flag bit is used to atomically block new references via
pmap_extract_and_hold() while removing managed mappings of a page.
Thus, the reference count of a page is guaranteed not to increase if the
page is unbusied, unmapped, and the object's write lock is held.  As
a consequence of this, the page lock no longer protects a page's
identity; operations which move pages between objects are now
synchronized solely by the objects' locks.

The vm_page_wire() and vm_page_unwire() KPIs are changed.  The former
requires that either the object lock or the busy lock is held.  The
latter no longer has a return value and may free the page if it releases
the last reference to that page.  vm_page_unwire_noq() behaves the same
as before; the caller is responsible for checking its return value and
freeing or enqueuing the page as appropriate.  vm_page_wire_mapped() is
introduced for use in pmap_extract_and_hold().  It fails if the page is
concurrently being unmapped, typically triggering a fallback to the
fault handler.  vm_page_wire() no longer requires the page lock and
vm_page_unwire() now internally acquires the page lock when releasing
the last wiring of a page (since the page lock still protects a page's
queue state).  In particular, synchronization details are no longer
leaked into the caller.

The change excises the page lock from several frequently executed code
paths.  In particular, vm_object_terminate() no longer bounces between
page locks as it releases an object's pages, and direct I/O and
sendfile(SF_NOCACHE) completions no longer require the page lock.  In
these latter cases we now get linear scalability in the common scenario
where different threads are operating on different files.

__FreeBSD_version is bumped.  The DRM ports have been updated to
accomodate the KPI changes.

Reviewed by:	jeff (earlier version)
Tested by:	gallatin (earlier version), pho
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20486
2019-09-09 21:32:42 +00:00
Johannes Lundberg
f6668e9f56 LinuxKPI: Improve sysfs support.
- Add functions for creating and merging sysfs groups.
- Add sysfs_streq function to compare strings ignoring newline from the
  sysctl userland call.
- Add a call to sysfs_create_groups in device_add.
- Remove duplicate header include.
- Bump __FreeBSD_version.

Reviewed by:	hselasky
Approved by:	imp (mentor), hselasky
MFC after:	4 days
Differential Revision:	D21542
2019-09-06 15:43:53 +00:00
Johannes Lundberg
458ba18d43 LinuxKPI: Add sysfs create/remove functions that handles multiple files in one call.
Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
Differential Revision:	D21475
2019-09-02 14:51:59 +00:00
Hans Petter Selasky
4d83500fda Use DEVICE memory instead of UNCACHEABLE on aarch64 in ioremap() in the LinuxKPI.
This fixes system hangs on reading device registers on aarch64.

Tested with:	Marvell MACCHIATObin (Armada8k) + mlx4en, amdgpu
Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D20789
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-09-02 08:34:45 +00:00
Conrad Meyer
419fe172c2 Linuxkpi: Prevent easy generated ctor name conflicts with prefix
Sponsored by:	Dell EMC Isilon
2019-08-17 03:00:58 +00:00
Hans Petter Selasky
4f109faadc Implement pci_enable_msi() and pci_disable_msi() in the LinuxKPI.
This patch makes the DRM graphics driver in ports usable on aarch64.

Submitted by:	Greg V <greg@unrelenting.technology>
Differential Revision:	https://reviews.freebsd.org/D21008
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-08-14 09:36:25 +00:00
John Baldwin
e28024fbad Fix build with DRM and INVARIANTS enabled.
The DRM drivers use the lockdep assertion macros with spinlock_t locks
which are backed by mutexes, not sx locks.  This causes compile
failures since you can't use sx_assert with a mutex.  Instead, change
the lockdep macros to use lock_class methods.  This works by assuming
that each LinuxKPI locking primitive embeds a FreeBSD lock as its
first structure and uses a cast to get to the underlying 'struct
lock_object'.

Reviewed by:	hselasky
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20992
2019-08-13 21:15:59 +00:00
Justin Hibbits
4eaa2fde6f Fix 32-bit build again, post r350570.
Missed this part with my testing as well.  Pass the right type to
BUS_TRANSLATE_RESOURCE().
2019-08-04 20:00:39 +00:00
Justin Hibbits
4b238da67b Fix 32-bit build post-r350570
The error message prints a rman_res_t, which is an uintmax_t.  Explicitly
cast, just for future-proofing, and use the correct format.
2019-08-04 19:55:43 +00:00
Justin Hibbits
937a05ba81 Add necessary bits for Linux KPI to work correctly on powerpc
PowerPC, and possibly other architectures, use different address ranges for
PCI space vs physical address space, which is only mapped at resource
activation time, when the BAR gets written.  The DRM kernel modules do not
activate the rman resources, soas not to waste KVA, instead only mapping
parts of the PCI memory at a time.  This introduces a
BUS_TRANSLATE_RESOURCE() method, implemented in the Open Firmware/FDT PCI
driver, to perform this necessary translation without activating the
resource.

In addition to system KPI changes, LinuxKPI is updated to handle a
big-endian host, by adding proper endian swaps to the I/O functions.

Submitted by:	mmacy
Reported by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D21096
2019-08-04 19:28:10 +00:00
Mark Johnston
eeacb3b02f Merge the vm_page hold and wire mechanisms.
The hold_count and wire_count fields of struct vm_page are separate
reference counters with similar semantics.  The remaining essential
differences are that holds are not counted as a reference with respect
to LRU, and holds have an implicit free-on-last unhold semantic whereas
vm_page_unwire() callers must explicitly determine whether to free the
page once the last reference to the page is released.

This change removes the KPIs which directly manipulate hold_count.
Functions such as vm_fault_quick_hold_pages() now return wired pages
instead.  Since r328977 the overhead of maintaining LRU for wired pages
is lower, and in many cases vm_fault_quick_hold_pages() callers would
swap holds for wirings on the returned pages anyway, so with this change
we remove a number of page lock acquisitions.

No functional change is intended.  __FreeBSD_version is bumped.

Reviewed by:	alc, kib
Discussed with:	jeff
Discussed with:	jhb, np (cxgbe)
Tested by:	pho (previous version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D19247
2019-07-08 19:46:20 +00:00
Hans Petter Selasky
8996977a89 Remove dead code added after r348743 in the LinuxKPI. The
LINUXKPI_VERSION macro is not defined for any compiled LinuxKPI code
which basically means __GFP_NOTWIRED is never checked when allocating
pages. This should work fine with the existing external DRM code as
long as the page wiring and unwiring is balanced.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-07-03 09:48:20 +00:00
Johannes Lundberg
6425fed7e6 LinuxKPI: Additions to rcu list.
- Add rcu list functions.
- Make rcu hlist's foreach macro use rcu calls instead of the non-rcu macro.
- Bump FreeBSD version so we have a checkpoint for the vboxvideo drm driver.

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
Differential Revision:	D20719
2019-06-21 18:48:07 +00:00
Johannes Lundberg
62260f68b4 LinuxKPI: Add atomic_long_sub macro.
Reviewed by:	imp (mentor), hps
Approved by:	imp (mentor), hps
MFC after:	1 week
Differential Revision:	D20718
2019-06-21 16:43:16 +00:00
Mark Johnston
88ea538a98 Replace uses of vm_page_unwire(m, PQ_NONE) with vm_page_unwire_noq(m).
These calls are not the same in general: the former will dequeue the
page if it is enqueued, while the latter will just leave it alone.  But,
all existing uses of the former apply to unmanaged pages, which are
never enqueued in the first place.  No functional change intended.

Reviewed by:	kib
MFC after:	1 week
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20470
2019-06-07 18:23:29 +00:00
Mark Johnston
1ef5e651fd Make the linuxkpi's alloc_pages() consistently return wired pages.
Previously it did this only on platforms without a direct map.  This
also more closely matches Linux's semantics.

Since some DRM v5.0 code assumes the old behaviour, use a
LINUXKPI_VERSION guard to preserve that until the out-of-tree module
is updated.

Reviewed by:	hselasky, kib (earlier versions), johalun
MFC after:	1 week
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20502
2019-06-06 16:09:19 +00:00
Conrad Meyer
e12be3218a Include eventhandler.h in more compilation units
This was enumerated with exhaustive search for sys/eventhandler.h includes,
cross-referenced against EVENTHANDLER_* usage with the comm(1) utility.  Manual
checking was performed to avoid redundant includes in some drivers where a
common os_bsd.h (for example) included sys/eventhandler.h indirectly, but it is
possible some of these are redundant with driver-specific headers in ways I
didn't notice.

(These CUs did not show up as missing eventhandler.h in tinderbox.)

X-MFC-With:	r347984
2019-05-21 01:18:43 +00:00
Johannes Lundberg
6a65ca35dd LinuxKPI: Finalize import of seq_file.
seq_file.h and linux_seq_file.c was imported form ports earlier but
linux_seq_file.c was never compiled in with the module. With this
commit base seq_file will replace ports seq_file and it required a
few modifications to not break functionality and build.

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-16 21:17:18 +00:00
Johannes Lundberg
6da2681fbc LinuxKPI: Add in_task macro.
This patch is part of D19565

Reviewed by:	hps, bwidawsk
Approved by:	imp (mentor), hps
Obtained from:	bwidawsk
MFC after:	1 week
2019-05-16 21:07:37 +00:00
Johannes Lundberg
39881afcba LinuxKPI: Fix build on powerpc/sparc.
Use cmpset instead of testandset in tasklet lock code.

Reviewed by:	hps
Approved by:	imp (mentor), hps
Obtained from:	hps
MFC after:	1 week
2019-05-16 19:32:11 +00:00
Johannes Lundberg
480995dcf0 LinuxKPI: Updates to tasklets for Linux 5.0.
DRM drivers expect tasklets to have a counter for enable/disable calls.
Also, add a few more tasklet locking functions.

This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-16 18:03:08 +00:00
Johannes Lundberg
07e0a3ca50 LinuxKPI: Add group_leader member to struct task_struct.
Assign self as group leader at creation to act as the only member of a
new process group.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-16 17:53:36 +00:00
Johannes Lundberg
47e2723ad7 LinuxKPI: Update access_ok macro for v5.0.
Check LINUXKPI_VERSION macro for backwards compatibility.
It's recommended to update any drivers that depend on the older KPI
so we can deprecate < 5.0 code as we update to newer Linux version.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-16 17:44:17 +00:00
Tycho Nightingale
b961c0f244 Allow loading the same DMA address multiple times without any prior
unload for the LinuxKPI.

Reviewed by:	kib, zeising
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20181
2019-05-16 17:41:16 +00:00
Johannes Lundberg
c4e0746e7d LinuxKPI: Add helper macros IS_ALIGNED and DIV_ROUND_DOWN_ULL.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 17:57:06 +00:00
Johannes Lundberg
0bb30b3a19 LinuxKPI: Move {lower|upper}_32_bits macros from port to base.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 17:48:11 +00:00
Johannes Lundberg
8264104401 LinuxKPI: Include asm/atomic-long.h from atomic.h.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 17:44:25 +00:00
Johannes Lundberg
d109700cf0 LinuxKPI: Add get_random_u32 function.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 17:32:00 +00:00
Johannes Lundberg
3137d2d4ec LinuxKPI: Update user_access_begin for Linux v5.0.
Check the new LINUXKPI_VERSION macro for backwards compatibility.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 17:04:12 +00:00
Johannes Lundberg
a4a9f2267e LinuxKPI: Expand ktime functionality.
Also, make ktime_get_raw call getnanouptime instead of getnanotime
to match (the correct) ktime_get_raw_ns.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-15 16:59:04 +00:00
Johannes Lundberg
65ff7a3192 LinuxKPI: Add prepare to pm_ops and bump FreeBSD version.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-14 23:50:46 +00:00
Johannes Lundberg
1462308d8b LinuxKPI: Add vm_fault_t type.
This patch is part of D19565

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-14 23:32:02 +00:00
Johannes Lundberg
395be823fd LinuxKPI: Add context member to ww_mutex and bump FreeBSD version.
This patch is part of https://reviews.freebsd.org/D19565.

Reviewed by:	hps
Approved by:	imp (mentor), hps
2019-05-14 23:21:20 +00:00
Johannes Lundberg
02927c768a LinuxKPI: Let del_timer return a value to match Linux.
This patch is part of https://reviews.freebsd.org/D19565.

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-14 23:12:14 +00:00
Johannes Lundberg
5098ed5f3b Implement linux_pci_unregister_drm_driver in linuxkpi so that drm drivers
can be unloaded.

This patch is a part of D19565.

Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
2019-05-10 23:10:22 +00:00
Hans Petter Selasky
e2eb11e577 Fix memory leak of PCI BUS structure in the LinuxKPI.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-05-09 10:23:42 +00:00
Hans Petter Selasky
eb6f534241 Fix regression issue after r346645 in the LinuxKPI.
Make sure LinuxKPI PCI devices get a default BUSDMA tag.

Found by:	Thomas Laus <lausts@acm.org>
Sponsored by:	Mellanox Technologies
2019-05-09 09:45:19 +00:00
Hans Petter Selasky
423530be04 Add support for Dynamic Interrupt Moderation, DIM, in mlx5en(4).
Add support for DIM based on Linux,
with some minor adaptions specific to FreeBSD.

Linux commit
f97c3dc3c0e8d23a5c4357d182afeef4c67f5c33

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-05-08 10:23:33 +00:00
Hans Petter Selasky
46068e86c3 Use PCIV_INVALID in pci_channel_offline() in the LinuxKPI.
Build tested drm-current-kmod prior to commit.

MFC after:		1 week
Submitted by:		slavash@
Sponsored by:		Mellanox Technologies
2019-05-06 16:22:45 +00:00
Hans Petter Selasky
fa23397925 Disabling a PCI device should only disable busmaster in the LinuxKPI.
As Linux comment for this function point:
Signal to the system that the PCI device is not in use by the system
anymore. This only involves disabling PCI bus-mastering, if active.

Build tested drm-current-kmod prior to commit.

MFC after:		1 week
Submitted by:		slavash@
Sponsored by:		Mellanox Technologies
2019-05-06 16:17:38 +00:00
Hans Petter Selasky
34cb771e01 Implement print_hex_dump_debug() function macro in the LinuxKPI.
Build tested drm-current-kmod prior to commit.

MFC after:		1 week
Submitted by:		slavash@
Sponsored by:		Mellanox Technologies
2019-05-06 16:10:26 +00:00
Hans Petter Selasky
4580f5eadd Allow controlling pr_debug at runtime in the LinuxKPI.
Turning on pr_debug at compile time make it non-optional at runtime.
This often means that the amount of the debugging is unbearable.
Allow developer to turn on pr_debug output only when needed.

Build tested drm-current-kmod prior to commit.

MFC after:		1 week
Submitted by:		kib@
Sponsored by:		Mellanox Technologies
2019-05-06 16:00:20 +00:00
Hans Petter Selasky
442d12d89c Fix regression issue after r346645 in the LinuxKPI.
The S/G list must be mapped AS-IS without any optimisations.
This also implies that sg_dma_len() must be equal to sg->length.
Many Linux drivers assume this and this fixes some DRM issues.

Put the BUS DMA map pointer into the scatter-gather list to
allow multiple mappings on the same physical memory address.

The FreeBSD version has been bumped to force recompilation of
external kernel modules.

Sponsored by:		Mellanox Technologies
2019-05-04 09:47:01 +00:00
Hans Petter Selasky
8ec9f0282a Fix regression issue after r346645 in the LinuxKPI.
Properly handle error case when mapping DMA address fails.

Sponsored by:		Mellanox Technologies
2019-05-04 09:30:03 +00:00
Hans Petter Selasky
a6619e8d9c Reduce the number of mutexes after r346645 in the LinuxKPI.
Make function macro wrappers for locking and unlocking to ease readability.

No functional change.

Discussed with:		kib@, tychon@ and zeising@
Sponsored by:		Mellanox Technologies
2019-04-30 10:41:20 +00:00
Hans Petter Selasky
93a203ea65 Make the dma_pool structure private to the LinuxKPI similar to Linux.
No functional change.

Discussed with:		kib @
Sponsored by:		Mellanox Technologies
2019-04-30 09:38:22 +00:00
Hans Petter Selasky
5a637529ed Store a pointer to the device instead of the PCI device in the DMA pool
implementation in the LinuxKPI. This avoids use of container_of().

No functional change.

Discussed with:		kib @
Sponsored by:		Mellanox Technologies
2019-04-30 09:26:11 +00:00
Johannes Lundberg
af248a7cee Don't call cdev_init where cdev_alloc is called. cdev_alloc already
handles initialization.

Reported by:	johalun
Reviewed by:	hps
Approved by:	imp (mentor), hps
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D19565
2019-04-25 21:54:32 +00:00
Tycho Nightingale
b09626b330 LinuxKPI buildfix for ppc64 after r346645.
Proposed by:	hselasky
Sponsored by:	Dell EMC Isilon
2019-04-25 18:13:55 +00:00
Hans Petter Selasky
d4fedb75ec LinuxKPI buildfix for 32-bit DMA architectures after r346645.
The <sys/pctrie.h> APIs expect a 64-bit DMA key.
This is fine as long as the DMA is less than or equal to 64 bits, which
is currently the case.

Sponsored by:		Mellanox Technologies
2019-04-25 09:13:15 +00:00
Tycho Nightingale
f211d536b6 LinuxKPI should use bus_dma(9) to be compatible with an IOMMU
Reviewed by:	hselasky, kib
Tested by:	greg@unrelenting.technology
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19845
2019-04-24 20:30:45 +00:00
Ed Maste
ff9be73ee3 Enable ioremap for aarch64 in the LinuxKPI
Required for Mellanox drivers (e.g. on Ampere eMAG at Packet.com).

PR:		237055
Submitted by:	Greg V <greg@unrelenting.technology>
Reviewed by:	hselasky
Differential Revision:	https://reviews.freebsd.org/D19987
2019-04-20 15:57:05 +00:00
Conrad Meyer
a8a16c7128 Replace read_random(9) with more appropriate arc4rand(9) KPIs
Reviewed by:	ae, delphij
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19760
2019-04-04 01:02:50 +00:00
Hans Petter Selasky
582141f69d Revert r345102 until the DRM next port issues are resolved.
Requested by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-03-14 09:18:54 +00:00
Hans Petter Selasky
7d595f6b79 Resolve duplicate symbol name conflict after r345095, when building LINT.
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-03-13 19:53:20 +00:00
Hans Petter Selasky
998f22eb4b Implement sg_virt() function in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:31:33 +00:00
Hans Petter Selasky
56b16627d4 Define SG_CHAIN and SG_END in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:30:40 +00:00
Hans Petter Selasky
c469882529 Implement pr_info_ratelimited() function macro in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:26:24 +00:00
Hans Petter Selasky
856b815d27 Define some RCU debug macros in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:24:30 +00:00
Hans Petter Selasky
0ac26c0e30 Honor SYSCTL function return values when creating sysfs nodes in the LinuxKPI.
Return proper error code upon failure.

Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:21:19 +00:00
Hans Petter Selasky
925245aaa9 Implement more malloc function macros in the LinuxKPI.
Fix arguments for currently unused kvmalloc().

Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:17:52 +00:00
Hans Petter Selasky
ab62989ae9 Implement more PCI speed related functions and macros in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:15:36 +00:00
Hans Petter Selasky
75f7460b34 Implement IS_ALIGNED() and DIV_ROUND_DOWN_ULL() function macros in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:04:06 +00:00
Hans Petter Selasky
8734a56285 Implement si_meminfo() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 19:01:55 +00:00
Hans Petter Selasky
5746b1cd46 Implement task_euid() and get_task_state() function macros in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:55:41 +00:00
Hans Petter Selasky
c7486758eb Implement get_task_comm() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:53:29 +00:00
Hans Petter Selasky
638fa5a36f Implement current_exiting() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:51:33 +00:00
Hans Petter Selasky
845a91ce0b Implement list_for_each_entry_from_reverse() and
list_bulk_move_tail() in the LinuxKPI.

Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:47:17 +00:00
Hans Petter Selasky
4f081c4fc6 Implement dma_map_page_attrs() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:44:06 +00:00
Hans Petter Selasky
839b4bf24d Implement ida_free() and ida_alloc_max() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:02:47 +00:00
Hans Petter Selasky
8b2a8a4954 Implement DEFINE_STATIC_SRCU() function macro in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 18:00:25 +00:00
Hans Petter Selasky
884aaac660 Implement BITS_PER_TYPE() function macro in the LinuxKPI.
Fix some style while at it.

Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 17:55:58 +00:00
Hans Petter Selasky
4c09177ab7 Properly define the DMA attribute values in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 17:51:08 +00:00
Hans Petter Selasky
7f36930024 Implement dev_err_once() function macro in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 17:46:05 +00:00
Hans Petter Selasky
8fdb5febfc Implement dma_set_mask_and_coherent() in the LinuxKPI.
Submitted by:		Johannes Lundberg <johalun0@gmail.com>
MFC after:		1 week
Sponsored by:		Limelight Networks
Sponsored by:		Mellanox Technologies
2019-03-13 17:42:31 +00:00