1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
dependent encoding) to NEWCONS (Unicode).
The file "LANG.map" is used to convert INDEX.keymaps. It has 3 columns:
- the language ID as used in the source file
- the language ID to be used in the generated file (e.g. "iw" -> "he")
- the encoding of the menu texts for this language
The conversion result is written to STDOUT.
The file "KBDFILES.map" is used to batch convert keymap files. It's
columns are:
- the encoding used for the keymap sounce file
- the name of the source file
- the name of the generated file
The output files are created in the TEMP sub-directory of the vt keymap
directory, in order to preserve (possibly uncommitted) keymap files in
/usr/src/share/vt/keymaps.
The convert-keymap.pl script can be directly executed by passing the
source file name and the encoding on the command line. It writes to
STDOUT and generates hex Unicode codepoints by default. (This can be
changed to decimal in the script.)
While written for the one-time conversion of the SYSCONS keymaps into
the format required for NEWCONS, I think these tools may be useful for
easy conversion of possible further SYSCONS keymap files, that have not
been committed to the source tree.
Mostly bugfixes or features developed in the past 6 months,
so this is a 10.1 candidate.
Basically no user API changes (some bugfixes in sys/net/netmap_user.h).
In detail:
1. netmap support for virtio-net, including in netmap mode.
Under bhyve and with a netmap backend [2] we reach over 1Mpps
with standard APIs (e.g. libpcap), and 5-8 Mpps in netmap mode.
2. (kernel) add support for multiple memory allocators, so we can
better partition physical and virtual interfaces giving access
to separate users. The most visible effect is one additional
argument to the various kernel functions to compute buffer
addresses. All netmap-supported drivers are affected, but changes
are mechanical and trivial
3. (kernel) simplify the prototype for *txsync() and *rxsync()
driver methods. All netmap drivers affected, changes mostly mechanical.
4. add support for netmap-monitor ports. Think of it as a mirroring
port on a physical switch: a netmap monitor port replicates traffic
present on the main port. Restrictions apply. Drive carefully.
5. if_lem.c: support for various paravirtualization features,
experimental and disabled by default.
Most of these are described in our ANCS'13 paper [1].
Paravirtualized support in netmap mode is new, and beats the
numbers in the paper by a large factor (under qemu-kvm,
we measured gues-host throughput up to 10-12 Mpps).
A lot of refactoring and additional documentation in the files
in sys/dev/netmap, but apart from #2 and #3 above, almost nothing
of this stuff is visible to other kernel parts.
Example programs in tools/tools/netmap have been updated with bugfixes
and to support more of the existing features.
This is meant to go into 10.1 so we plan an MFC before the Aug.22 deadline.
A lot of this code has been contributed by my colleagues at UNIPI,
including Giuseppe Lettieri, Vincenzo Maffione, Stefano Garzarella.
MFC after: 3 days.
Remove the .t wrappers
Rename all of the TAP test applications from test-<test> to
<test>_test to match the convention described in the TestSuite
wiki page
humanize_number_test.c:
- Fix -Wformat warnings with counter variables
- Fix minor style(9) issues:
-- Header sorting
-- Variable declaration alignment/sorting in main(..)
-- Fit the lines in <80 columns
- Fix an off by one index error in the testcase output [*]
- Remove unnecessary `extern char * optarg;` (this is already provided by
unistd.h)
Phabric: D555
Approved by: jmmv (mentor)
MFC after: 2 weeks
Obtained from: EMC / Isilon Storage Division [*]
Submitted by: Casey Peel <cpeel@isilon.com> [*]
Sponsored by: EMC / Isilon Storage Division
The new code uses a "test discovery mechanism" to determine
what tests are available for execution
The test shell can be specified via:
kyua test -v test_suites.FreeBSD.bin.sh.test_shell=/path/to/test/sh
Sponsored by: EMC / Isilon Storage Division
Approved by: jmmv (mentor)
Reviewed by: jilles (maintainer)
submitted via r268811
- Install the Kyuafile by adding FILES to FILESGROUPS
- Run the testcases with an unprivileged user
Some of the testcases depend upon behavior that's broken when
run as root on FreeBSD because of how permissions are treated
with access(2) vs eaccess(2), open(2), etc
- Simplify the test driver to just inspect the exit code from
run_test because it now exits with 0 if successful and exits
with !0 if unsuccessful
- Don't do ad hoc temporary directory creation/deletion; let Kyua
handle that
- Add entries for files removed in r268811 to
OptionalObsoleteFiles.inc
PR: 191020
X-MFC with: r268811
Approved by: jmmv (mentor)
Reviewed by: bapt
Sponsored by: EMC / Isilon Storage Division
* Add AR9287 and AR9331 chipset support - it just uses the AR5416 support
for now so some of the register mappings are wrong, but it works well
enough.
* don't call exit() from opmark().
Rename all of the TAP test applications from <test> to <test>_test
to match the convention described in the TestSuite wiki page
Phabric: D538
Approved by: jmmv (mentor)
Sponsored by: EMC / Isilon Storage Division
- Make sure the tests go into the right directory. The location was
wrong so they were overwriting the bin/chown tests!
- Use the right naming scheme for the test program.
- Remove the svn:executable property from the shell script.
variants. This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.
Exp-run: antoine
PR: 189842
Discussed with: bapt
Sponsored by: DARPA, AFRL
This is a redo of r267424, which was reverted in r267432 because it broke
"make buildworld" from FreeBSD 9.x. This issue has been resolved and this
change is still "make tinderbox" clean.
At least one test doesn't work yet without gcc, however gcc is
not always available in base. Using the environment compiler
is more trustable and will also work with an external compiler.
Reviewed by: markj
MFC after: 3 days
statically linked into consumers (GDB and variants) in the base
system, and the shared library is no longer installed.
That also allows ports to be able to use a modern version of readline
PR: 162948
Reviewed by: emaste
The test locale1.0 depends on locale support; it is meaningless without a
working LC_MESSAGES.
I added an OptionalObsoleteFiles.inc entry.
PR: 181151
Submitted by: Garrett Cooper (original version)
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
use dedicated kernel files with some local settings
use mkimg for ISO building
put images into separate directory and rename them for better consistency
The _SUPPORT knobs have a consistent meaning which differs from the
behaviour controlled by this knob. As the knob is opt-out and has not
appeared in a release the impact should be low.
Suggested by: imp, wblock
MFC after: 1 week
* Add special case handling where normal conversion would not work
(some APIs have special names)
* Fix conversion for function calls involving ifnet
Submitted by: Sreekanth Rupavatharam <rupavath@juniper.net>
Obtained from: Juniper Networks, Inc.
This was never intended to be off by default but was done this way
because the initial patches broke the build. Things seem stable now
(have been so for a while too) and "make tinderbox" is clean so let's
try again.
Announced in freebsd-current; all reported shortcomings have been
addressed.
vtfontcvt is useful for end users to convert arbitrary bitmap fonts
for use by vt(4). It can also be used as a build tool, allowing us
to keep the source font data in the src tree rather than uuencoded
binaries.
Reviewed by: ray, wblock (D183)
Sponsored by: The FreeBSD Foundation
Perform an O(n) deduplication pass over the bold maps at the end, rather
than walking the normal map list to look for a duplicate glyph each time
a bold mapping entry is added.
Sponsored by: The FreeBSD Foundation
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
In r266650, we made libatf-c and libatf-c++ private libraries so that no
components outside of the source tree could unintendedly depend on them.
This change does the same for the "atf-sh library" by moving the atf-sh
interpreter from its public location in /usr/bin/ to the private location
in /usr/libexec/. Our build system will ensure that our own test programs
use the right binary, but users won't be able to depend on atf-sh by
"mistake".
Committing this now to ride the UPDATING notice added with r267172 today.
The libatf-* major version numbers in FreeBSD were one version ahead of
upstream because, when atf was first imported into FreeBSD, the upstream
numbers were not respected. This is just confusing and bound to cause
problems down the road.
Fix this by taking advantage of the fact that libatf-* are now private
and that atf is not yet built by default. However, and unfortunately, a
clean build is needed for tests to continue working once "make
delete-old-libs" has been run; hence the note in UPDATING.
The GNU Unifont .hex format is a text file. Each line represents one
glyph and consists of a four-digit hex code point, a colon, and pairs of
hex digits representing the bitmap. By default an 8x16 font is assumed,
with 16x16 double-width glyphs, resulting in either 32 or 64 hex digits
for the bitmap.
Our version of the file format supports comments at the top of the file
to set the height and width:
# Height: <decimal height>
# Width: <decimal width>
Each row of bitmap data is rounded up to byte width - for example, a
10-pixel wide font uses 4 characters per row.
See http://czyborra.com/unifont/ for more background on the original
format.
Sponsored by: The FreeBSD Foundation
Walking a linked list of all glyphs to look for a duplicate is very slow
for large fonts (e.g., for CJK character sets). In my test the runtime
for a sample 40000 character font went from just over 80 seconds on
average to just over 2 seconds.
Sponsored by: The FreeBSD Foundation
The description of WITH/WITHOUT tweaks should only document the non-default
option. TESTS still defaults to no, so the option to be documented is
WITH_TESTS.
to use the procedural interface.
Submitted by: Sreekanth Rupavatharam <rupavath@juniper.net>
Reviewed by: glebius@
Obtained from: Juniper Networks, Inc.
Update the manpage to reflect this change.
- Always set the current position to the first null-byte when opening in append
mode. This makes the implementation compatible with glibc's. Update the test
suite.
Reported by: pho
Approved by: cognet
We should not be leaking these interfaces to the outside world given
that it's much easier for third-party components to use the devel/atf
package from ports.
As a side-effect, we can also drop the ATF pkgconfig and aclocal files
from the base system. Nothing in the base system needs these, and it
was quite ugly to have to get them installed only so that a few ports
could build. The offending ports have been fixed to depend on
devel/atf explicitly.
Reviewed by: bapt
the configuration can reference additional files relative to its own
location.
(NANO_MODULES): If set to "default", install all built modules.
Reviewed by: imp
MFC after: 1 week
install it as fmake. This defaults to no. This should be viewed as the
first step towards evental migration of this historic code to ports
and removal from the tree.
to bus_space(9) and that uses the proto(4) driver for talking to
hardware. If the I/O resource is a memory mapped I/O resource,
then mmap(2) will be attempted to avoid read(2)/write(2) overhead.
Sponsored by: Juniper Networks, Inc.
option. Convert all other uses to MK_CTF=no. Set MK_CTF=no rather than
the indirect WITHOUT_CDDL in filemon regression. It is expected that
NO_CTF will be removed in FreeBSD 12 entirely.
An execute-only fd (opened with O_EXEC) allows neither read() nor write()
and is therefore incompatible with all stdio modes. Therefore, the [EINVAL]
error applies.
Also adjust the similar check in freopen() with a NULL path, even though
this checks an fd which is already from a FILE.
building clang and/or gcc as the bootstrap compiler. Normally, the
default compiler is used. WITH_CLANG_BOOTSTRAP and/or
WITH_GCC_BOOTSTRAP will enable building these compilers as part
bootstrap phase. WITH/WITHOUT_CLANG_IS_CC controls which compiler is
used by default for the bootstrap phase, as well as which compiler is
installed as cc. buildworld now successfully completes building the
cross compiler with WITHOUT_CLANG=t and WITHOUT_GCC=t and produces a
built system with neither of these included.
Similarlly, MK_BINUTILS_BOOTSTRAP controls whether binutils is built
during this phase.
WITHOUT_CROSS_COMPILER will now force MK_BINUTILS_BOOTSTRAP=no,
MK_CLANG_BOOTSTRAP=no and MK_GCC_BOOTSTRAP=no.
BOOTSTRAP_COMPILER was considered, but rejected, since pc98 needs both
clang and gcc to bootstrap still. It should be revisisted in the
future if this requirement goes away. Values should be gcc, clang or
none. It could also be a list.
The odd interaction with Xfoo cross/external tools needs work, but
is beyond the scope of this change as well.
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
issues with vendors that needed 7.x support have been resolved. Many
vendors are still using 8.x build platforms, however, so bumping this
up to 9.0 will have to wait until that is resolved. Actual support for
building from 8.x still relies on those vendors fixing bugs that are
present as most developers have moved onto 9.x or newer platforms.
Reviewed by: marcel@
Interestingly, the pkill tool lives in bin, not usr.bin. Haven't bothered
to check if this is because the tool moved or because the tests were
originally added in the wrong place.
Note that these tests are for fmake, not bmake, and thus they are not
installed nor run when bmake is selected (the default). Yes, I have
wasted a *ton* of time on moving tests for no real reason other than
ensuring they are not left behind.
But maybe, just maybe, it was not work in vain: the majority of these
tests also work with bmake and the few that don't may point at broken
stuff. For example, the tests for the "archive" feature do not work
with bmake, but bmake's manpage and source tree seem to imply that they
should. So... to be investigated later; need to poke sjg@.
I'm starting with the easy cases. The leftovers need to be looked at a
bit more closely.
Note that this change _does_ modify the code of the old tests. This is
required in order to allow the code to locate the data files in the
source directory instead of the current directory, because Kyua
automatically changes the latter to a temporary directory.
Also note that at least one test is known to be broken here. Actually,
the test is not really broken: it's marked as a TODO but unfortunately
Kyua's TAP parser currently does not understand that. Will have to be
fixed separately.
This change was originally going to only migrate the usr.sbin tests but, as
it turns out, the usr.sbin/sa/ tests require files from usr.bin/lastcomm/
so it's better to just also migrate the latter at the same time. The other
usr.bin tests will be moved separately.
To make these tests work within the test suite, some of them have required
changes to prevent modifying the source directory and instead just rely on
the current directory for file manipulation.
/cfg updated with the modified configuration files in /etc. I have
written an improved version with the following features:
* Recurses directories.
* Only requires file arguments the first time the file/directory is
* added to /cfg.
* Handles file deletions.
PR: 145962, 157533
Submitted by: Aragon Gouveia and Alex Bakhtin
AppleTalk was a network transport protocol for Apple Macintosh devices
in 80s and then 90s. Starting with Mac OS X in 2000 the AppleTalk was
a legacy protocol and primary networking protocol is TCP/IP. The last
Mac OS X release to support AppleTalk happened in 2009. The same year
routing equipment vendors (namely Cisco) end their support.
Thus, AppleTalk won't be supported in FreeBSD 11.0-RELEASE.
IPX was a network transport protocol in Novell's NetWare network operating
system from late 80s and then 90s. The NetWare itself switched to TCP/IP
as default transport in 1998. Later, in this century the Novell Open
Enterprise Server became successor of Novell NetWare. The last release
that claimed to still support IPX was OES 2 in 2007. Routing equipment
vendors (e.g. Cisco) discontinued support for IPX in 2011.
Thus, IPX won't be supported in FreeBSD 11.0-RELEASE.
interface, in the r241616 a crutch was provided. It didn't work well, and
finally we decided that it is time to break ABI and simply make if_baudrate
a 64-bit value. Meanwhile, the entire struct if_data was reviewed.
o Remove the if_baudrate_pf crutch.
o Make all fields of struct if_data fixed machine independent size. The
notion of data (packet counters, etc) are by no means MD. And it is a
bug that on amd64 we've got a 64-bit counters, while on i386 32-bit,
which at modern speeds overflow within a second.
This also removes quite a lot of COMPAT_FREEBSD32 code.
o Give 16 bit for the ifi_datalen field. This field was provided to
make future changes to if_data less ABI breaking. Unfortunately the
8 bit size of it had effectively limited sizeof if_data to 256 bytes.
o Give 32 bits to ifi_mtu and ifi_metric.
o Give 64 bits to the rest of fields, since they are counters.
__FreeBSD_version bumped.
Discussed with: emax
Sponsored by: Netflix
Sponsored by: Nginx, Inc.
This function is not public and brooks (initial committer adding the code)
suggests the deletion of the tests (which I don't know if they work)
instead of changing the visibility of the function.
Just replace the simple calls to the library with ad-hoc code. We should
later rewrite these with the ATF libraries anyway, which are part of the
base system.
Tests that cannot be run because a precondition is not met should be
marked as skipped, not failed. Do this for the tests in mdconfig that
first check if the caller user is root.
