Freescale's QorIQ line includes a new ethernet controller, based on their
Datapath Acceleration Architecture (DPAA). This uses a combination of a Frame
manager, Buffer manager, and Queue manager to improve performance across all
interfaces by being able to pass data directly between hardware acceleration
interfaces.
As part of this import, Freescale's Netcomm Software (ncsw) driver is imported.
This was an attempt by Freescale to create an OS-agnostic sub-driver for
managing the hardware, using shims to interface to the OS-specific APIs. This
work was abandoned, and Freescale's primary work is in the Linux driver (dual
BSD/GPL license). Hence, this was imported directly to sys/contrib, rather than
going through the vendor area. Going forward, FreeBSD-specific changes may be
made to the ncsw code, diverging from the upstream in potentially incompatible
ways. An alternative could be to import the Linux driver itself, using the
linuxKPI layer, as that would maintain parity with the vendor-maintained driver.
However, the Linux driver has not been evaluated for reliability yet, and may
have issues with the import, whereas the ncsw-based driver in this commit was
completed by Semihalf 4 years ago, and is very stable.
Other SoC modules based on DPAA, which could be added in the future:
* Security and Encryption engine (SEC4.x, SEC5.x)
* RAID engine
Additional work to be done:
* Implement polling mode
* Test vlan support
* Add support for the Pattern Matching Engine, which can do regular expression
matching on packets.
This driver has been tested on the P5020 QorIQ SoC. Others listed in the
dtsec(4) manual page are expected to work as the same DPAA engine is included in
all.
Obtained from: Semihalf
Relnotes: Yes
Sponsored by: Alex Perez/Inertial Computing
Summary:
Many instances of bus_alloc_resource() simply use 0 and ~0 for start and end to
denote 'anywhere' with a given count. To clean this up, introduce a
bus_alloc_resource_anywhere() convenience function.
Bump __FreeBSD_version for the new API.
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D5370
This allows 'make analyze' or 'make OBJ.clang-analyzer' to run the
Clang static analyzer and present results on stdout.
Obtained from: NetBSD (CVS Rev. 1.3)
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D5449
- bsd.progs.mk is safe to include regardless of PROGS/PROGS_CXX/SCRIPTS
being set.
- bsd.progs.mk includes bsd.prog.mk always and will bring in
bsd.files.mk and bsd.obj.mk.
- DIRDEPS_BUILD: There's no need for _SKIP_BUILD or _SKIP_STAGING as the
PROGS were implicitly being built by the staging dependency anyway. This
was also preventing other objects from building if they were not part of
the staging sets.
- DIRDEPS_BUILD: This fixes PROGS without bsd.test.mk not staging.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
For PROGS this was recursing twice since MAKELEVEL0 is for 'dirdeps'
which then really builds in a sub-make.
Sponsored by: EMC / Isilon Storage Division
r96164 added them to avoid recursing twice with _SUBDIR. That issue was
fixed in bsd.subdir.mk in r291635 for all targets.
Sponsored by: EMC / Isilon Storage Division
As of r295661 elfcopy supports PE format for EFI boot binaries and is a
viable objcopy implementation for the base system and ports.
The (temporary) src.conf knob WITHOUT_ELFCOPY_AS_OBJCOPY knob may be set
to obtain the GNU version if necessary.
PR: 207091 [exp-run]
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
The OBJS_DEPEND_GUESS mechanism required moving the bsd.dep.mk inclusion
to after the checks, but left DEPENDFILE not-yet-set. Move it to
bsd.own.mk to resolve this.
Pointyhat to: bdrewery
Reported by: antoine (ports failures)
Sponsored by: EMC / Isilon Storage Division
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.
Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277
Given PROG1 PROG2, 'make PROG1' would work but 'make PROG1 PROG2' would not.
Just build them as normal in a sub-make to avoid any issues.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
For example when building, from buildworld, lib/atf/libatf-c++/tests/detail:
--- all_subdir_atf ---
is now:
--- all_subdir_lib/atf/libatf-c++/tests/detail ---
Sponsored by: EMC / Isilon Storage Division
This will generate dependencies rather than depending on the previous behavior
of depending on the guessed OBJS: *.h dependecies or a user running
'make depend'.
Experimentation showed that depending only on headers was not enough and
prone to .ORDER errors. Downstream users may also have added
dependencies into beforedepend or afterdepend targets. The safest way to
ensure dependencies are generated before build is to run 'make depend'
beforehand rather than just depending on DPSRCS+SRCS.
Note that the OBJS_DEPEND_GUESS mechanism (a.k.a .if !exists(.depend) then
foo.o: *.h) is still useful as it improves incremental builds with missing
.depend.* files and allows 'make foo.o' to usually work, while this
'beforebuild: depend' ensures that the build will always find all dependencies.
The 'make foo.o' case has no means of a 'beforebuild' hook.
This also removes several hacks in the DIRDEPS_BUILD:
- NO_INSTALL_INCLUDES is no longer needed as it mostly was to work around
.ORDER problems with building the needed headers early.
- DIRDEPS_BUILD: It is no longer necesarry to track "local dependencies" in
Makefile.depend.
These were only in Makefile.depend for 'clean builds' since nothing would
generate the files due to skipping 'make depend' and early dependency
bugs that have been fixed, such as adding headers into SRCS for the
OBJS_DEPEND_GUESS mechanism. Normally if a .depend file does not exist then
a dependency is added by bsd.lib.mk/bsd.prog.mk from OBJS: *.h. However,
meta.autodep.mk creates a .depend file from created meta files and inserts
that into Makefile.depend. It also only tracks *.[ch] files though which can
miss some dependencies that are hooked into 'make depend'. This .depend
that is created then breaks incremental builds due to the !exists(.depend)
checks for OBJS_DEPEND_GUESS. The goal was to skip 'make depend' yet it only
really works the first time. After that files are not generated as expected,
which r288966 tried to address but was using buildfiles: rather than
beforebuild: and was reverted in r291725. As noted previously,
depending only on headers in beforebuild: would create .ORDER errors
in some cases.
meta.autodep.mk is still used to generate Makefile.depend though via:
gendirdeps: Makefile.depend
.END: gendirdeps
This commit allows removing all of the "local dependencies" in
Makefile.depend which cuts down on churn and removes some of the
arch-dependent Makefile.depend files.
The "local dependencies" were also problematic for bootstrapping.
Sponsored by: EMC / Isilon Storage Division
FAST_DEPEND is intended to be the "skip 'make depend' and mkdep"
feature. Since DIRDEPS_BUILD does this already with some of its own
hacks, and filemon doesn't need this, and nofilemon does, teach it how
to handle each of these cases.
In meta+filemon mode filemon will handle dependencies itself via the
meta mode logic in bmake. We still want to set MK_FAST_DEPEND=yes to
enable some logic that indicates that 'make depend' is skipped in the
traditional sense. The actual .depend.* files will be skipped.
When nofilemon is set though we still need to track and generate dependencies.
Sponsored by: EMC / Isilon Storage Division
The .depend file will still be generated if _EXTRADEPEND is used. The target
is kept with a dependency on DPSRCS though so that 'make depend' will generate
all files.
Sponsored by: EMC / Isilon Storage Division
Rather than depend on .depend not existing, check the actual
.depend.OBJ file that will be used for that object. If it doesn't
exist then use the guessed dependencies.
FAST_DEPEND may never have a .depend file. Not having one means all of the
previous logic would over-depend all object files on all headers which is not
what we wanted. It also means that if a .depend is generated before a build
is done for _EXTRADEPEND (such as for PROG or LIB) then all of these
dependencies would not be used since the .depend wasn't generated from mkdep
and the real .depend.* files are not generated until the build.
Sponsored by: EMC / Isilon Storage Division
cleandepend should always remove CLEANDEPEND* if they are not empty,
but bsd.dep.mk should not add the tags entries unless SRCS is defined
as it did before. The .depend file itself it still always removed
to avoid accidentally keeping a stale one around as done in r295666.
Sponsored by: EMC / Isilon Storage Division
is exhausted.
How to use:
Basically we need to add on rc.conf an another option like:
If we want to protect only the main processes.
syslogd_oomprotect="YES"
If we want to protect all future children of the specified processes.
syslogd_oomprotect="ALL"
PR: 204741 (based on)
Submitted by: eugen@grosbein.net
Reviewed by: jhb, allanjude, rpokala and bapt
MFC after: 4 weeks
Relnotes: Yes
Sponsored by: gandi.net
Differential Revision: https://reviews.freebsd.org/D5176
and geom_uncompress(4):
1. mkuzip(8):
- Proper support for eliminating all-zero blocks when compressing an
image. This feature is already supported by the geom_uzip(4) module
and CLOOP format in general, so it's just a matter of making mkuzip(8)
match. It should be noted, however that this feature while it sounds
great, results in very slight improvement in the overall compression
ratio, since compressing default 16k all-zero block produces only 39
bytes compressed output block, which is 99.8% compression ratio. With
typical average compression ratio of amd64 binaries and data being
around 60-70% the difference between 99.8% and 100.0% is not that
great further diluted by the ratio of number of zero blocks in the
uncompressed image to the overall number of blocks being less than
0.5 (typically). However, this may be important from performance
standpoint, so that kernel are not spinning its wheels decompressing
those empty blocks every time this zero region is read. It could also
be important when you create huge image mostly filled with zero
blocks for testing purposes.
- New feature allowing to de-duplicate output image. It turns out that
if you twist CLOOP format a bit you can do that as well. And unlike
zero-blocks elimination, this gives a noticeable improvement in the
overall compression ratio, reducing output image by something like
3-4% on my test UFS2 3GB image consisting of full FreeBSD base system
plus some of the packages (openjdk, apache etc), about 2.3GB worth of
file data (800+MB compressed). The only caveat is that images created
with this feature "on" would not work on older versions of FeeBSDxi
kernel, hence it's turned off by default.
- provide options to control both features and document them in manual
page.
- merge in all relevant LZMA compression support from the mkulzma(8),
add new option to select between both.
- switch license from ad-hoc beerware into standard 2-clause BSD.
2. geom_uzip(4):
- implement support for de-duplicated images;
- optimize some code paths to handle "all-zero" blocks without reading
any compressed data;
- beef up manual page to explain that geom_uzip(4) is not limited only
to md(4) images. The compressed data can be written to the block
device and accessed directly via magic of GEOM(4) and devfs(4),
including to mount root fs from a compressed drive.
- convert debug log code from being compiled in conditionally into
being present all the time and provide two sysctls to turn it on or
off. Due to intended use of the module, it can be used in
environments where there may not be a luxury to put new kernel with
debug code enabled. Having those options handy allows debug issues
without as much problem by just having access to serial console or
network shell access to a box/appliance. The resulting additional
CPU cycles are just few int comparisons and branches, and those are
minuscule when compared to data decompression which is the main
feature of the module.
- hopefully improve robustness and resiliency of the geom_uzip(4) by
performing some of the data validation / range checking on the TOC
entries and rejecting to attach to an image if those checks fail.
- merge in all relevant LZMA decompression support from the
geom_uncompress(4), enable automatically when appropriate format is
indicated in the header.
- move compilation work into its own worker thread so that it does not
clog g_up. This allows multiple instances work in parallel utilizing
smp cores.
- document new knobs in the manual page.
Reviewed by: adrian
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D5333
and manual page were.
For whatever reason it listed myself as a primary author, which is
just not true.
Also, majority of the manpage is copied verbatim from the geom_uzip(4),
contributed by ceri, with only minor adjustments from loos, so put ceri
back into the copyright secrion where he belongs and reflect that in the
AUTHORS section.
For what it's worth, I think this one should be deleted and LZMA
support just folded back into geom_uzip(4) / mkuzip(4) whete it really
belongs.
MFC after: 1 month
This will speed up some tree-walks with FAST_DEPEND which otherwise
would include length(SRCS) .depend files.
This also uses a trick suggested by sjg@ to still read them in when
specifying _V_READ_DEPEND=1 in the env/make args.
Sponsored by: EMC / Isilon Storage Division
While "jng" or "jib" without arguments told you what each sub-command does,
sub-command usage didn't tell you (e.g., "jng bridge" or "jib addm" gave
only usage and not description).
While here clean up the documentation for jail_list
PR: 196152
Approved by: jamie, wblock
MFC after: 1 week, with r295471
Differential Revision: https://reviews.freebsd.org/D5243
Reduce differences between jib/jng and fix a bug that would prevent
additional interfaces from being created if the first of many already
existed (counter wasn't incremented before calling only continue).
Adding `!' before an interface name will disable MAC allocation, falling
back to driver mechanics. Alternatively adding `=' before an interface name
causes the MAC address to be cloned (for ng_bridge(4) back-end only). While
here, disable the auto-detection of wlan* since this knocks the host off;
requiring the host that defines the jail to explicitly enable this feature
by preceding the interface with `='.
Documented in iwn(4), "Only one virtual interface may be configured at any
time." However, netgraph with a cloned MAC address is able to communicate
over an ng_eiface attached to an ng_bridge linked to the wlan(4) interface.
While here, introduce syntax to specify the MAC address is to be cloned if
the named interface begins with equals [=].
As of r294068 boot1.efi can load loader.efi from ZFS.
As of r295320 boot1.efi prefers to load loader.efi from the same device
it was loaded from.
MFC after: 3 days
Sponsored by: The FreeBSD Foundation
Replace `make regress` (legacy test make target) and `make test` (incomplete
test make target added with the FreeBSD test suite) with make check as it's
consistent with other open source projects.
`make check` defaults to running tests from `.OBJDIR`, but can be overridden
with the `CHECKDIR` variable.
Add `make checkworld` target to simplify running the FreeBSD test suite from
`TESTSBASE` (i.e. the top-level tests directory), similar to buildworld.
Document `make check` and `make checkworld` in build(7).
Other minor changes:
- Rename intermediate file (`Kyuafile.auto`) to `Kyuafile` to simplify
`make check`.
- Remove terse warnings attached to `beforetest`/`aftertest`.
- Add kyua binary check to check target in suite.test.mk; error out if it's
not found
The MFC is [partly] contingent on other build related changes being MFCed.
Differential Revision: https://reviews.freebsd.org/D4406
MFC after: 2 months
X-MFC to: stable/10
Relnotes: yes
Reviewed by: bdrewery, Evan Cramer <eccramer@gmail.com>
Sponsored by: EMC / Isilon Storage Division
The NVMe specification does not define a maximum or optimal delete
size, so technically max delete size is min(full size of namespace,
2^32 - 1 LBAs). A single delete operation for a multi-TB NVMe
namespace though may take much longer to complete than the nvme(4)
I/O timeout period. So choose a sensible default here that is still
suitably large to minimize the number of overall delete operations.
This also fixes possible uint32_t overflow on initial TRIM operation
for zpool create operations for NVMe namespaces with >4G LBAs.
MFC after: 3 days
Sponsored by: Intel
This hack will be removed in a few weeks. It is here to fix incremental
builds of SSH between r291941 and r294370.
Reported by: jmallett
MFC after: 1 day
Sponsored by: EMC / Isilon Storage Division
Summary:
Migrate to using the semi-opaque type rman_res_t to specify rman resources. For
now, this is still compatible with u_long.
This is step one in migrating rman to use uintmax_t for resources instead of
u_long.
Going forward, this could feasibly be used to specify architecture-specific
definitions of resource ranges, rather than baking a specific integer type into
the API.
This change has been broken out to facilitate MFC'ing drivers back to 10 without
breaking ABI.
Reviewed By: jhb
Sponsored by: Alex Perez/Inertial Computing
Differential Revision: https://reviews.freebsd.org/D5075
Several attempts to fix this logic was done after r241298, which were
all reverted, yet this change was not.
The .h file does not depend on the .c file, so do not impose such a
dependency on it. They are generated by the same command but do not
depend on each other. Restore the .ORDER which should handle parallel build
issues. This fixes an actual bug where the .h file is not recreated
when missing [1]. For example:
cd lib/libc
make cleanobj
make nsparser.h
rm nsparser.h
make nsparser.h # will not rebuild nsparser.h
I have been trying to track down a build problem where nsparser.h is
missing when nslexer.o is built. It is possible this is related.
Reported by: bde [1]
https://lists.freebsd.org/pipermail/svn-src-all/2012-October/059481.htmlhttps://lists.freebsd.org/pipermail/svn-src-all/2012-October/060038.html
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
DIRDEPS_BUILD does not yet support PROGS having their own dependency
file.
Overriding .MAKE.DEPENDFILE here causes major problems with the meta
mode logic since it creates the Makefile.depend as '.depend' resulting
in infinite loops in make due to dirdeps.mk including .depend endlessly.
X-MFC-With: r294752
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
We have had this user-modifable DEPENDFILE variable forever that does nothing
relevant for the user since fmake always used '.depend'. Bmake
introduced the .MAKE.DEPENDFILE variable that can be modified to change
the name of '.depend'.
Prior to r284288, bsd.progs.mk was setting .MAKE.DEPENDFILE to allow
working incremental builds. This was modified most likely to not
conflict with the META MODE handling of .MAKE.DEPENDFILE as it has a lot
more special logic for that variable.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Currently dtrace(1) -Go does not properly rebuild the target if it
exists. It results in missing symbols.
dtrace -C -x nolibs -G -o usdt.o -s /root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/usdt.d tst.usdt.o
dtrace: target object (usdt.o) already exists. Please remove the target
dtrace: object and rebuild all the source objects if you wish to run the DTrace
dtrace: linking process again
cc -O2 -pipe -O0 -g -I/root/git/freebsd/cddl/usr.sbin/dtrace/tests/common/json -std=gnu99 -fstack-protector-strong -Qunused-arguments -o tst.usdt.exe.full tst.usdt.o usdt.o
tst.usdt.o: In function `main':
/root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:56: undefined reference to `__dtrace_bunyan_fake___log__debug'
/root/git/freebsd/cddl/contrib/opensolaris/cmd/dtrace/test/tst/common/json/tst.usdt.c:60: undefined reference to `__dtrace_bunyan_fake___log__debug'
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [tst.usdt.exe.full] Error code 1
This is a consequence of r212358.
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
sent using roundrobin protocol and set a better granularity and distribution
among the interfaces. Tuning the number of packages sent by interface can
increase throughput and reduce unordered packets as well as reduce SACK.
Example of usage:
# ifconfig bge0 up
# ifconfig bge1 up
# ifconfig lagg0 create
# ifconfig lagg0 laggproto roundrobin laggport bge0 laggport bge1 \
192.168.1.1 netmask 255.255.255.0
# ifconfig lagg0 rr_limit 500
Reviewed by: thompsa, glebius, adrian (old patch)
Approved by: bapt (mentor)
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D540
control algorithm options. The argument is variable length and is opaque
to TCP, forwarded directly to the algorithm's ctl_output method.
Provide new includes directory netinet/cc, where algorithm specific
headers can be installed.
The new API doesn't yet have any in tree consumers.
The original code written by lstewart.
Reviewed by: rrs, emax
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D711
