This is a MFC into stable/10 of:
- r258286 Update notes for imports of atf.
- r258289 MFV: Import atf-0.18.
- r258290 Drop all ATF tools code.
This is "make tinderbox" clean as run on ref10-amd64 with the default
WITHOUT_TESTS option. A "make buildworld" with WITH_TESTS set now works
as well.
This is a MFC into stable/10 of:
- r257849 Add libatf-c++ to the prebuild libs.
- r257853 Build and install the atf tests.
- r258233 Move all atf directories to the tests mtree.
- r258285 Fix the build of some ATF tests.
This change is "make tinderbox" clean on ref10-amd64 with the default
settings of WITHOUT_TESTS. It is likely for the WITH_TESTS build to
still be broken because not all relevant changes have been merged yet.
This is a MFC of the following into stable/10:
- r257097 Set up the /usr/tests hierarchy.
- r257098 Add missing WITHOUTTESTS file.
- r257100 Add a tests(7) manual page.
- r257105 Disable WITHTESTS= for now.
- r257848 Fix buildworld when WITHTESTS is enabled.
- r257850 Subsume the functionality of MKATF into MKTESTS.
- r257851 Handle the removal of the test suite when WITHOUTTESTS=yes.
- r257852 Install category Kyuafiles from their category directories.
- r258232 Install BSD.tests.mtree when MKTESTS is yes.
Note that building with WITH_TESTS is still broken at this point (and
hence why WITHOUT_TESTS is the set as the default). Subsequent pullups
will fix the remaining issues.
Consistently add the relocation offset only when the ELF type is not
ET_EXEC. This fixes several problems with the DTrace pid provider not
being able to match probes.
Modify the libproc breakpoint add/remove functions to stop the target
process if it has not already been stopped, since this is required for
ptrace(2) to work.
libdtrace does not seem to stop target processes before trying to remove
their breakpoints, so we were previously failing to remove the breakpoint
on r_debug_state() in rtld. This was causing processes to die with SIGTRAP
if they called dlopen(3) after dtrace(1) had detached.
This is a MFC of:
- r256761 Clearly split the logic to build ATF and plain tests apart.
- r256762 Add the automatic generation of Atffile files.
- r256763 Add the automatic generation of Kyuafile files.
- r256764 Plug atf-run into the 'test' target.
- r256765 Plug kyua into the 'test' target.
- r257096 Move the TESTSBASE definition to bsd.own.mk.
- r257099 Add missing plain.test.mk.
- r258297 Remove registration of C++ test programs into PROGS.
- r258298 Fix the build of plain test programs.
- r258551 Install plain.test.mk.
- r259208 Add tap.test.mk.
Approved by: rpaulo (mentor)
MFC r256661:
Fix the libproc build when DEBUG is defined.
MFC r257222:
Clean up the debug printing in libproc a bit. In particular:
* Don't print any error messages to stderr unless DEBUG is defined.
* Add a DPRINTFX macro for use when errno isn't set.
* Print the error string from libelf when appropriate.
MFC r257235:
Remove an incorrect debug printf.
MFC r257248:
Fix the build with gcc.
MFC r257298:
Revert r257248 and fix the problem in a way that doesn't violate style(9).
Do not force to run atexit handlers, which text comes from a dso owning
the handle passed to __cxa_finalize() but which are registered by other
dso, when the process is inside exit(3).
Implement extended LUN support. If PIM_EXTLUNS is set by a SIM, encode
the upper 32-bits of the LUN, if possible, into the target_lun field as
passed directly from the REPORT LUNs response. This allows extended LUN
support to work for all LUNs with zeros in the lower 32-bits, which covers
most addressing modes without breaking KBI. Behavior for drivers not
setting PIM_EXTLUNS is unchanged. No user-facing interfaces are modified.
Extended LUNs are stored with swizzled 16-bit word order so that, for
devices implementing LUN addressing (like SCSI-2), the numerical
representation of the LUN is identical with and without PIM_EXTLUNS. Thus
setting PIM_EXTLUNS keeps most behavior, and user-facing LUN IDs, unchanged.
This follows the strategy used in Solaris. A macro (CAM_EXTLUN_BYTE_SWIZZLE)
is provided to transform a lun_id_t into a uint64_t ordered for the wire.
This is the second part of work for full 64-bit extended LUN support and is
designed to a bridge for stable/10 to the final 64-bit LUN code. The
third and final part will involve widening lun_id_t to 64 bits and will
not be MFCed. This third part will break the KBI but will keep the KPI
unchanged so that all drivers that will care about this can be updated now
and not require code changes between HEAD and stable/10.
Reviewed by: scottl
Support SNI in libfetch
SNI is Server Name Indentification which is a protocol for TLS that
indicates the host that is being connected to at the start of the
handshake. It allows to use Virtual Hosts on HTTPS.
PR: kern/183583
Approved by: bapt (implicit)
Approved by: re (gjb)
r258537 (hrs):
Add ICONV_{GET,SET}_ILSEQ_INVALID iconvctl. GNU iconv returns
EILSEQ when there is an invalid character in the output codeset
while it is valid in the input. However, POSIX requires iconv()
to perform an implementation-defined conversion on the character.
So, Citrus iconv converts such a character to a special character
which means it is invalid in the output codeset.
This is not a problem in most cases but some software like libxml2
depends on GNU's behavior to determine if a character is output
as-is or another form such as a character entity (&#NNN;).
r258587 (peter):
Move the iconv wrapper source from libc_nonshared to libc/iconv so
that it is all in the one place again. Rename libc/iconv/iconv.c
to bsd_iconv.c. Compile the wrappers into libc.a so that
WITHOUT_DYNAMICROOT works again.
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation
Bug fixes in iconv(3) UTF-7 support.
- Add ' to the list of directly encoded characters and * to the list of
optionally directly encoded characters as per RFC 2152.
- In _citrus_UTF7_mbtoutf16 on end of input when the next output character
has only been partially decoded, save a copy of the buffer of input
characters (not just its length). On the next call with more input
characters this buffer is reprocessed together with the new input to
form a fully decoded output character.
- At the end of a base64 encoded sequence fully discard '-' (BASE64_OUT)
by decrementing psenc->chlen and i. This is needed to make room in
psenc->ch (input buffer) in case the next input character starts a new
base64 encoded sequence. And also, if this is the end of input and no
output character can be returned, this brings the encoder in the initial
state as indicated by _citrus_UTF7_stdenc_get_state_desc_generic which
is used by the caller to distinguish between no output and partial
output.
- In _citrus_UTF7_mbrtowc_priv pass the s parameter (input pointer)
directly to _citrus_UTF7_mbtoutf16 instead of a copy (s0). This way s
is updated correctly in case of errors.
- In _citrus_UTF7_mbrtowc_priv when called with psenc->surrogate set
(previous call did not have enough input), retrieve the previously
decoded UTF-16 character from (psenc->cache >> psenc->bits) instead of
(psenc->cache >> 2).
Approved by: re (kib)
r257583 (peter):
Remove the WITH_LIBICONV_COMPAT hack that seems to do more harm
than good. This caused libc to spoof the ports libiconv namespace
and provide a colliding libiconv.so.3 to fool rtld. This should
have been removed some time ago.
r258012:
Remove WITH_LIBICONV_COMPAT file to chase after r257583.
r258013:
Regenerate src.conf.5 after removal of WITH_LIBICONV_COMPAT.
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation
Sponsored by: The FreeBSD Foundation
- Use system libiconv, instead of trying to dlopen() it.
- Just disable recoding support in libsmb if built WITHOUT_ICONV.
PR: 183153
Approved by: re (kib)
- Add manual pages for capability rights (rights(4)), cap_rights_init(3)
family of functions and cap_rights_get(3) function.
- Update remaining Capsicum-related manual pages.
Sponsored by: The FreeBSD Foundation
Reviewed by: bdrewery
Approved by: re (glebius)
Make getutxent(3) more robust against bad utx.log files. Whenever we read
zeroes, don't stop processing the file, but read until its end or valid
data.
In collaboration with: ed
Approved by: re (kib)
Drop cm_lock before calling mapper_close, which in turn could call
_citrus_mapper_close again and result in a deadlock otherwise.
This is similar to NetBSD PR/24023 (fixed in their r1.5 of this file).
PR: bin/182994
Submitted by: Fabian Keil <fk fabiankeil de>
Approved by: re (hrs)
Add a new capability, VM_CAP_ENABLE_INVPCID, that can be enabled to expose
'invpcid' instruction to the guest. Currently bhyve will try to enable this
capability unconditionally if it is available.
Consolidate code in bhyve to set the capabilities so it is no longer
duplicated in BSP and AP bringup.
Add a sysctl 'vm.pmap.invpcid_works' to display whether the 'invpcid'
instruction is available.
Approved by: re (hrs)
Only accept 'net' and 'pxe' devices as underlying transport
in tftp.c on x86. Prior to this change tftp code would attempt
to send packets over any boot device, including zfs one with
predictably sad results.
Approved by: re (gjb)
MFC After: 1 month
Since so many programs don't check return value, always NUL terminate
the buf...
fix rounding when using base 1024 (the bug that started it all)...
add a set of test cases so we can make sure that things don't break
in the future...
Thanks to Clifton Royston for testing and the test program...
Approved by: re (hrs, glebius)
MFC after: 1 week
Make the amd64/pmap code aware of nested page table mappings used by bhyve
guests. This allows bhyve to associate each guest with its own vmspace and
deal with nested page faults in the context of that vmspace. This also
enables features like accessed/dirty bit tracking, swapping to disk and
transparent superpage promotions of guest memory.
Guest vmspace:
Each bhyve guest has a unique vmspace to represent the physical memory
allocated to the guest. Each memory segment allocated by the guest is
mapped into the guest's address space via the 'vmspace->vm_map' and is
backed by an object of type OBJT_DEFAULT.
pmap types:
The amd64/pmap now understands two types of pmaps: PT_X86 and PT_EPT.
The PT_X86 pmap type is used by the vmspace associated with the host kernel
as well as user processes executing on the host. The PT_EPT pmap is used by
the vmspace associated with a bhyve guest.
Page Table Entries:
The EPT page table entries as mostly similar in functionality to regular
page table entries although there are some differences in terms of what
bits are used to express that functionality. For e.g. the dirty bit is
represented by bit 9 in the nested PTE as opposed to bit 6 in the regular
x86 PTE. Therefore the bitmask representing the dirty bit is now computed
at runtime based on the type of the pmap. Thus PG_M that was previously a
macro now becomes a local variable that is initialized at runtime using
'pmap_modified_bit(pmap)'.
An additional wrinkle associated with EPT mappings is that older Intel
processors don't have hardware support for tracking accessed/dirty bits in
the PTE. This means that the amd64/pmap code needs to emulate these bits to
provide proper accounting to the VM subsystem. This is achieved by using
the following mapping for EPT entries that need emulation of A/D bits:
Bit Position Interpreted By
PG_V 52 software (accessed bit emulation handler)
PG_RW 53 software (dirty bit emulation handler)
PG_A 0 hardware (aka EPT_PG_RD)
PG_M 1 hardware (aka EPT_PG_WR)
The idea to use the mapping listed above for A/D bit emulation came from
Alan Cox (alc@).
The final difference with respect to x86 PTEs is that some EPT implementations
do not support superpage mappings. This is recorded in the 'pm_flags' field
of the pmap.
TLB invalidation:
The amd64/pmap code has a number of ways to do invalidation of mappings
that may be cached in the TLB: single page, multiple pages in a range or the
entire TLB. All of these funnel into a single EPT invalidation routine called
'pmap_invalidate_ept()'. This routine bumps up the EPT generation number and
sends an IPI to the host cpus that are executing the guest's vcpus. On a
subsequent entry into the guest it will detect that the EPT has changed and
invalidate the mappings from the TLB.
Guest memory access:
Since the guest memory is no longer wired we need to hold the host physical
page that backs the guest physical page before we can access it. The helper
functions 'vm_gpa_hold()/vm_gpa_release()' are available for this purpose.
PCI passthru:
Guest's with PCI passthru devices will wire the entire guest physical address
space. The MMIO BAR associated with the passthru device is backed by a
vm_object of type OBJT_SG. An IOMMU domain is created only for guest's that
have one or more PCI passthru devices attached to them.
Limitations:
There isn't a way to map a guest physical page without execute permissions.
This is because the amd64/pmap code interprets the guest physical mappings as
user mappings since they are numerically below VM_MAXUSER_ADDRESS. Since PG_U
shares the same bit position as EPT_PG_EXECUTE all guest mappings become
automatically executable.
Thanks to Alan Cox and Konstantin Belousov for their rigorous code reviews
as well as their support and encouragement.
Thanks for John Baldwin for reviewing the use of OBJT_SG as the backing
object for pci passthru mmio regions.
Special thanks to Peter Holm for testing the patch on short notice.
Approved by: re
Discussed with: grehan
Reviewed by: alc, kib
Tested by: pho
The accept(2) man page warns that O_NONBLOCK and other properties on the
new socket may vary across implementations. However, this issue only
applies to accept() and not to accept4(). On the other hand, accept4()
is not commonly available yet.
Reported by: pluknet
Reviewed by: bjk
Approved by: re (kib)
for. This is useful for software needing to know which architecture a
binary is built for as arm and armv6 have slight differences meaning only
some binaries build for one will work as expected on the other. It is
expected pkgng will be able to make use of this to simplify the logic to
determine which package ABI to use.
Approved by: re (kib)
This connects LLDB to the build, but it is disabled by default. Add
WITH_LLDB= to src.conf to build it.
Note that LLDB requires a C++11 compiler so is disabled on platforms
using GCC.
Approved by: re (gjb)
Sponsored by: DARPA, AFRL
exhausted.
- Add a new protect(1) command that can be used to set or revoke protection
from arbitrary processes. Similar to ktrace it can apply a change to all
existing descendants of a process as well as future descendants.
- Add a new procctl(2) system call that provides a generic interface for
control operations on processes (as opposed to the debugger-specific
operations provided by ptrace(2)). procctl(2) uses a combination of
idtype_t and an id to identify the set of processes on which to operate
similar to wait6().
- Add a PROC_SPROTECT control operation to manage the protection status
of a set of processes. MADV_PROTECT still works for backwards
compatability.
- Add a p_flag2 to struct proc (and a corresponding ki_flag2 to kinfo_proc)
the first bit of which is used to track if P_PROTECT should be inherited
by new child processes.
Reviewed by: kib, jilles (earlier version)
Approved by: re (delphij)
MFC after: 1 month
preprocessor) gives the following error:
--- Version.map ---
<stdin>:287:4: error: invalid preprocessing directive
# Implemented as weak aliases for imprecise versions
^
1 error generated.
Change the comment to a C-style one, to prevent this error.
Approved by: re (hrs)
an address in the first 2GB of the process's address space. This flag should
have the same semantics as the same flag on Linux.
To facilitate this, add a new parameter to vm_map_find() that specifies an
optional maximum virtual address. While here, fix several callers of
vm_map_find() to use a VMFS_* constant for the findspace argument instead of
TRUE and FALSE.
Reviewed by: alc
Approved by: re (kib)
