Since m_cat() may copy data from the second mbuf chain into the last mbuf
of the first chain, it may free the first mbuf of the second chain. Thus,
the second chain is not guaranteed to be valid after m_cat() returns.
Reviewed by: glebius
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D5497
This is done to prevent not having CCACHE_DIR causing meta mode with filemon to
see stat changes in the ccache dir and cause rebuilds.
Sponsored by: EMC / Isilon Storage Division
improve cancellation robustness.
Introduce a new file operation, fo_aio_queue, which is responsible for
queueing and completing an asynchronous I/O request for a given file.
The AIO subystem now exports library of routines to manipulate AIO
requests as well as the ability to run a handler function in the
"default" pool of AIO daemons to service a request.
A default implementation for file types which do not include an
fo_aio_queue method queues requests to the "default" pool invoking the
fo_read or fo_write methods as before.
The AIO subsystem permits file types to install a private "cancel"
routine when a request is queued to permit safe dequeueing and cleanup
of cancelled requests.
Sockets now use their own pool of AIO daemons and service per-socket
requests in FIFO order. Socket requests will not block indefinitely
permitting timely cancellation of all requests.
Due to the now-tight coupling of the AIO subsystem with file types,
the AIO subsystem is now a standard part of all kernels. The VFS_AIO
kernel option and aio.ko module are gone.
Many file types may block indefinitely in their fo_read or fo_write
callbacks resulting in a hung AIO daemon. This can result in hung
user processes (when processes attempt to cancel all outstanding
requests during exit) or a hung system. To protect against this, AIO
requests are only permitted for known "safe" files by default. AIO
requests for all file types can be enabled by setting the new
vfs.aio.enable_usafe sysctl to a non-zero value. The AIO tests have
been updated to skip operations on unsafe file types if the sysctl is
zero.
Currently, AIO requests on sockets and raw disks are considered safe
and are enabled by default. aio_mlock() is also enabled by default.
Reviewed by: cem, jilles
Discussed with: kib (earlier version)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D5289
taskqueue_enqueue() was changed to support both fast and non-fast
taskqueues 10 years ago in r154167. It has been a compat shim ever
since. It's time for the compat shim to go.
Submitted by: Howard Su <howard0su@gmail.com>
Reviewed by: sephe
Differential Revision: https://reviews.freebsd.org/D5131
Freescale's QorIQ line includes a new ethernet controller, based on their
Datapath Acceleration Architecture (DPAA). This uses a combination of a Frame
manager, Buffer manager, and Queue manager to improve performance across all
interfaces by being able to pass data directly between hardware acceleration
interfaces.
As part of this import, Freescale's Netcomm Software (ncsw) driver is imported.
This was an attempt by Freescale to create an OS-agnostic sub-driver for
managing the hardware, using shims to interface to the OS-specific APIs. This
work was abandoned, and Freescale's primary work is in the Linux driver (dual
BSD/GPL license). Hence, this was imported directly to sys/contrib, rather than
going through the vendor area. Going forward, FreeBSD-specific changes may be
made to the ncsw code, diverging from the upstream in potentially incompatible
ways. An alternative could be to import the Linux driver itself, using the
linuxKPI layer, as that would maintain parity with the vendor-maintained driver.
However, the Linux driver has not been evaluated for reliability yet, and may
have issues with the import, whereas the ncsw-based driver in this commit was
completed by Semihalf 4 years ago, and is very stable.
Other SoC modules based on DPAA, which could be added in the future:
* Security and Encryption engine (SEC4.x, SEC5.x)
* RAID engine
Additional work to be done:
* Implement polling mode
* Test vlan support
* Add support for the Pattern Matching Engine, which can do regular expression
matching on packets.
This driver has been tested on the P5020 QorIQ SoC. Others listed in the
dtsec(4) manual page are expected to work as the same DPAA engine is included in
all.
Obtained from: Semihalf
Relnotes: Yes
Sponsored by: Alex Perez/Inertial Computing
Summary:
Many instances of bus_alloc_resource() simply use 0 and ~0 for start and end to
denote 'anywhere' with a given count. To clean this up, introduce a
bus_alloc_resource_anywhere() convenience function.
Bump __FreeBSD_version for the new API.
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D5370
This allows 'make analyze' or 'make OBJ.clang-analyzer' to run the
Clang static analyzer and present results on stdout.
Obtained from: NetBSD (CVS Rev. 1.3)
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D5449
- bsd.progs.mk is safe to include regardless of PROGS/PROGS_CXX/SCRIPTS
being set.
- bsd.progs.mk includes bsd.prog.mk always and will bring in
bsd.files.mk and bsd.obj.mk.
- DIRDEPS_BUILD: There's no need for _SKIP_BUILD or _SKIP_STAGING as the
PROGS were implicitly being built by the staging dependency anyway. This
was also preventing other objects from building if they were not part of
the staging sets.
- DIRDEPS_BUILD: This fixes PROGS without bsd.test.mk not staging.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
For PROGS this was recursing twice since MAKELEVEL0 is for 'dirdeps'
which then really builds in a sub-make.
Sponsored by: EMC / Isilon Storage Division
r96164 added them to avoid recursing twice with _SUBDIR. That issue was
fixed in bsd.subdir.mk in r291635 for all targets.
Sponsored by: EMC / Isilon Storage Division
As of r295661 elfcopy supports PE format for EFI boot binaries and is a
viable objcopy implementation for the base system and ports.
The (temporary) src.conf knob WITHOUT_ELFCOPY_AS_OBJCOPY knob may be set
to obtain the GNU version if necessary.
PR: 207091 [exp-run]
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
The OBJS_DEPEND_GUESS mechanism required moving the bsd.dep.mk inclusion
to after the checks, but left DEPENDFILE not-yet-set. Move it to
bsd.own.mk to resolve this.
Pointyhat to: bdrewery
Reported by: antoine (ports failures)
Sponsored by: EMC / Isilon Storage Division
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.
Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277
Given PROG1 PROG2, 'make PROG1' would work but 'make PROG1 PROG2' would not.
Just build them as normal in a sub-make to avoid any issues.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
For example when building, from buildworld, lib/atf/libatf-c++/tests/detail:
--- all_subdir_atf ---
is now:
--- all_subdir_lib/atf/libatf-c++/tests/detail ---
Sponsored by: EMC / Isilon Storage Division
This will generate dependencies rather than depending on the previous behavior
of depending on the guessed OBJS: *.h dependecies or a user running
'make depend'.
Experimentation showed that depending only on headers was not enough and
prone to .ORDER errors. Downstream users may also have added
dependencies into beforedepend or afterdepend targets. The safest way to
ensure dependencies are generated before build is to run 'make depend'
beforehand rather than just depending on DPSRCS+SRCS.
Note that the OBJS_DEPEND_GUESS mechanism (a.k.a .if !exists(.depend) then
foo.o: *.h) is still useful as it improves incremental builds with missing
.depend.* files and allows 'make foo.o' to usually work, while this
'beforebuild: depend' ensures that the build will always find all dependencies.
The 'make foo.o' case has no means of a 'beforebuild' hook.
This also removes several hacks in the DIRDEPS_BUILD:
- NO_INSTALL_INCLUDES is no longer needed as it mostly was to work around
.ORDER problems with building the needed headers early.
- DIRDEPS_BUILD: It is no longer necesarry to track "local dependencies" in
Makefile.depend.
These were only in Makefile.depend for 'clean builds' since nothing would
generate the files due to skipping 'make depend' and early dependency
bugs that have been fixed, such as adding headers into SRCS for the
OBJS_DEPEND_GUESS mechanism. Normally if a .depend file does not exist then
a dependency is added by bsd.lib.mk/bsd.prog.mk from OBJS: *.h. However,
meta.autodep.mk creates a .depend file from created meta files and inserts
that into Makefile.depend. It also only tracks *.[ch] files though which can
miss some dependencies that are hooked into 'make depend'. This .depend
that is created then breaks incremental builds due to the !exists(.depend)
checks for OBJS_DEPEND_GUESS. The goal was to skip 'make depend' yet it only
really works the first time. After that files are not generated as expected,
which r288966 tried to address but was using buildfiles: rather than
beforebuild: and was reverted in r291725. As noted previously,
depending only on headers in beforebuild: would create .ORDER errors
in some cases.
meta.autodep.mk is still used to generate Makefile.depend though via:
gendirdeps: Makefile.depend
.END: gendirdeps
This commit allows removing all of the "local dependencies" in
Makefile.depend which cuts down on churn and removes some of the
arch-dependent Makefile.depend files.
The "local dependencies" were also problematic for bootstrapping.
Sponsored by: EMC / Isilon Storage Division
FAST_DEPEND is intended to be the "skip 'make depend' and mkdep"
feature. Since DIRDEPS_BUILD does this already with some of its own
hacks, and filemon doesn't need this, and nofilemon does, teach it how
to handle each of these cases.
In meta+filemon mode filemon will handle dependencies itself via the
meta mode logic in bmake. We still want to set MK_FAST_DEPEND=yes to
enable some logic that indicates that 'make depend' is skipped in the
traditional sense. The actual .depend.* files will be skipped.
When nofilemon is set though we still need to track and generate dependencies.
Sponsored by: EMC / Isilon Storage Division
The .depend file will still be generated if _EXTRADEPEND is used. The target
is kept with a dependency on DPSRCS though so that 'make depend' will generate
all files.
Sponsored by: EMC / Isilon Storage Division
Rather than depend on .depend not existing, check the actual
.depend.OBJ file that will be used for that object. If it doesn't
exist then use the guessed dependencies.
FAST_DEPEND may never have a .depend file. Not having one means all of the
previous logic would over-depend all object files on all headers which is not
what we wanted. It also means that if a .depend is generated before a build
is done for _EXTRADEPEND (such as for PROG or LIB) then all of these
dependencies would not be used since the .depend wasn't generated from mkdep
and the real .depend.* files are not generated until the build.
Sponsored by: EMC / Isilon Storage Division
cleandepend should always remove CLEANDEPEND* if they are not empty,
but bsd.dep.mk should not add the tags entries unless SRCS is defined
as it did before. The .depend file itself it still always removed
to avoid accidentally keeping a stale one around as done in r295666.
Sponsored by: EMC / Isilon Storage Division
is exhausted.
How to use:
Basically we need to add on rc.conf an another option like:
If we want to protect only the main processes.
syslogd_oomprotect="YES"
If we want to protect all future children of the specified processes.
syslogd_oomprotect="ALL"
PR: 204741 (based on)
Submitted by: eugen@grosbein.net
Reviewed by: jhb, allanjude, rpokala and bapt
MFC after: 4 weeks
Relnotes: Yes
Sponsored by: gandi.net
Differential Revision: https://reviews.freebsd.org/D5176
and geom_uncompress(4):
1. mkuzip(8):
- Proper support for eliminating all-zero blocks when compressing an
image. This feature is already supported by the geom_uzip(4) module
and CLOOP format in general, so it's just a matter of making mkuzip(8)
match. It should be noted, however that this feature while it sounds
great, results in very slight improvement in the overall compression
ratio, since compressing default 16k all-zero block produces only 39
bytes compressed output block, which is 99.8% compression ratio. With
typical average compression ratio of amd64 binaries and data being
around 60-70% the difference between 99.8% and 100.0% is not that
great further diluted by the ratio of number of zero blocks in the
uncompressed image to the overall number of blocks being less than
0.5 (typically). However, this may be important from performance
standpoint, so that kernel are not spinning its wheels decompressing
those empty blocks every time this zero region is read. It could also
be important when you create huge image mostly filled with zero
blocks for testing purposes.
- New feature allowing to de-duplicate output image. It turns out that
if you twist CLOOP format a bit you can do that as well. And unlike
zero-blocks elimination, this gives a noticeable improvement in the
overall compression ratio, reducing output image by something like
3-4% on my test UFS2 3GB image consisting of full FreeBSD base system
plus some of the packages (openjdk, apache etc), about 2.3GB worth of
file data (800+MB compressed). The only caveat is that images created
with this feature "on" would not work on older versions of FeeBSDxi
kernel, hence it's turned off by default.
- provide options to control both features and document them in manual
page.
- merge in all relevant LZMA compression support from the mkulzma(8),
add new option to select between both.
- switch license from ad-hoc beerware into standard 2-clause BSD.
2. geom_uzip(4):
- implement support for de-duplicated images;
- optimize some code paths to handle "all-zero" blocks without reading
any compressed data;
- beef up manual page to explain that geom_uzip(4) is not limited only
to md(4) images. The compressed data can be written to the block
device and accessed directly via magic of GEOM(4) and devfs(4),
including to mount root fs from a compressed drive.
- convert debug log code from being compiled in conditionally into
being present all the time and provide two sysctls to turn it on or
off. Due to intended use of the module, it can be used in
environments where there may not be a luxury to put new kernel with
debug code enabled. Having those options handy allows debug issues
without as much problem by just having access to serial console or
network shell access to a box/appliance. The resulting additional
CPU cycles are just few int comparisons and branches, and those are
minuscule when compared to data decompression which is the main
feature of the module.
- hopefully improve robustness and resiliency of the geom_uzip(4) by
performing some of the data validation / range checking on the TOC
entries and rejecting to attach to an image if those checks fail.
- merge in all relevant LZMA decompression support from the
geom_uncompress(4), enable automatically when appropriate format is
indicated in the header.
- move compilation work into its own worker thread so that it does not
clog g_up. This allows multiple instances work in parallel utilizing
smp cores.
- document new knobs in the manual page.
Reviewed by: adrian
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D5333
and manual page were.
For whatever reason it listed myself as a primary author, which is
just not true.
Also, majority of the manpage is copied verbatim from the geom_uzip(4),
contributed by ceri, with only minor adjustments from loos, so put ceri
back into the copyright secrion where he belongs and reflect that in the
AUTHORS section.
For what it's worth, I think this one should be deleted and LZMA
support just folded back into geom_uzip(4) / mkuzip(4) whete it really
belongs.
MFC after: 1 month
This will speed up some tree-walks with FAST_DEPEND which otherwise
would include length(SRCS) .depend files.
This also uses a trick suggested by sjg@ to still read them in when
specifying _V_READ_DEPEND=1 in the env/make args.
Sponsored by: EMC / Isilon Storage Division
While "jng" or "jib" without arguments told you what each sub-command does,
sub-command usage didn't tell you (e.g., "jng bridge" or "jib addm" gave
only usage and not description).
While here clean up the documentation for jail_list
PR: 196152
Approved by: jamie, wblock
MFC after: 1 week, with r295471
Differential Revision: https://reviews.freebsd.org/D5243
Reduce differences between jib/jng and fix a bug that would prevent
additional interfaces from being created if the first of many already
existed (counter wasn't incremented before calling only continue).
Adding `!' before an interface name will disable MAC allocation, falling
back to driver mechanics. Alternatively adding `=' before an interface name
causes the MAC address to be cloned (for ng_bridge(4) back-end only). While
here, disable the auto-detection of wlan* since this knocks the host off;
requiring the host that defines the jail to explicitly enable this feature
by preceding the interface with `='.
Documented in iwn(4), "Only one virtual interface may be configured at any
time." However, netgraph with a cloned MAC address is able to communicate
over an ng_eiface attached to an ng_bridge linked to the wlan(4) interface.
While here, introduce syntax to specify the MAC address is to be cloned if
the named interface begins with equals [=].
As of r294068 boot1.efi can load loader.efi from ZFS.
As of r295320 boot1.efi prefers to load loader.efi from the same device
it was loaded from.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
