Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.
sys/boot/efi/loader/Makefile
A LIBSTAND hack is no longer required for buildenv.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
file. That logic has grown quite significantly now;
o add a special handling for the snapshot images. Those have some
extra headers at the end of the image and we don't need those
in the output image really.
MFC after: 6 weeks
This change implements NOTE_ABSTIME flag for EVFILT_TIMER, which
specifies that the data field contains absolute time to fire the
event.
To make this useful, data member of the struct kevent must be extended
to 64bit. Using the opportunity, I also added ext members. This
changes struct kevent almost to Apple struct kevent64, except I did
not changed type of ident and udata, the later would cause serious API
incompatibilities.
The type of ident was kept uintptr_t since EVFILT_AIO returns a
pointer in this field, and e.g. CHERI is sensitive to the type
(discussed with brooks, jhb).
Unlike Apple kevent64, symbol versioning allows us to claim ABI
compatibility and still name the new syscall kevent(2). Compat shims
are provided for both host native and compat32.
Requested by: bapt
Reviewed by: bapt, brooks, ngie (previous version)
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D11025
This includes decoding both scheduler policy constants and the sched_param
structure for sched_get_priority_max(), sched_get_priority_min(),
sched_getparam(), sched_getscheduler(), sched_rr_get_interval(),
sched_setparam(), and sched_setscheduler().
- xohtml: Add "-w" option to pull support files from gh_pages
- Add "upload-xohtml-files" target to publish support files in gh_pages/
- add HISTORY/AUTHORS section to man pages
- xohtml: Add div.units as standard CSS text
- Don't treat values as format strings; they are not
- add "-p" to "mkdir -p build" in setup.sh
- add test case for {U:%%} (from df.c)
- detect end-of-string in '%' and '' escaping
- make xo_simple_field, for common simple cases
- xohtml: nuke "n" in "echo" commands
- rename "format" to "fmt" for consistency; same for "str" to "value"
- update test cases
Submitted by: phil
Based on feedback from OpenZFS developers Matt Ahrens and George Wilson,
the calculation of the ratio no longer takes in to account overhead.
The old formula could result in reporting a negative compression ratio
This could confuse the user or give a false impression that there would be
an advantage to disabling the compressed ARC feature.
The new formula will more closely match an average of the on-disk
compression ratio, as reported by the ZFS property 'compressratio'
MFC after: 3 days
Sponsored by: ScaleEngine Inc.
Tests that exercise the following flags are added in this commit:
- -A
- -H
- -I
- -g
- -h
- -k
- -m
Additional tests will be added soon.
MFC after: 1 month
These are were created proactively, in anticipation of the support being
fully implemented sometime in the future.
The tests currently fail on ^/head@r319845, however. Expect them to fail.
PR: 219933
Tested with: gdiff
Testcases for -H, -L, and -f haven't been implemented yet, in part due
to additional complexity needed to validate the features:
* -H and -f will require an external "helper" program to display/modify
the state/permissions for a given path.
* -L is being covered partially via the -n testcase today.
MFC after: 1 month
o Separate fields of struct socket that belong to listening from
fields that belong to normal dataflow, and unionize them. This
shrinks the structure a bit.
- Take out selinfo's from the socket buffers into the socket. The
first reason is to support braindamaged scenario when a socket is
added to kevent(2) and then listen(2) is cast on it. The second
reason is that there is future plan to make socket buffers pluggable,
so that for a dataflow socket a socket buffer can be changed, and
in this case we also want to keep same selinfos through the lifetime
of a socket.
- Remove struct struct so_accf. Since now listening stuff no longer
affects struct socket size, just move its fields into listening part
of the union.
- Provide sol_upcall field and enforce that so_upcall_set() may be called
only on a dataflow socket, which has buffers, and for listening sockets
provide solisten_upcall_set().
o Remove ACCEPT_LOCK() global.
- Add a mutex to socket, to be used instead of socket buffer lock to lock
fields of struct socket that don't belong to a socket buffer.
- Allow to acquire two socket locks, but the first one must belong to a
listening socket.
- Make soref()/sorele() to use atomic(9). This allows in some situations
to do soref() without owning socket lock. There is place for improvement
here, it is possible to make sorele() also to lock optionally.
- Most protocols aren't touched by this change, except UNIX local sockets.
See below for more information.
o Reduce copy-and-paste in kernel modules that accept connections from
listening sockets: provide function solisten_dequeue(), and use it in
the following modules: ctl(4), iscsi(4), ng_btsocket(4), ng_ksocket(4),
infiniband, rpc.
o UNIX local sockets.
- Removal of ACCEPT_LOCK() global uncovered several races in the UNIX
local sockets. Most races exist around spawning a new socket, when we
are connecting to a local listening socket. To cover them, we need to
hold locks on both PCBs when spawning a third one. This means holding
them across sonewconn(). This creates a LOR between pcb locks and
unp_list_lock.
- To fix the new LOR, abandon the global unp_list_lock in favor of global
unp_link_lock. Indeed, separating these two locks didn't provide us any
extra parralelism in the UNIX sockets.
- Now call into uipc_attach() may happen with unp_link_lock hold if, we
are accepting, or without unp_link_lock in case if we are just creating
a socket.
- Another problem in UNIX sockets is that uipc_close() basicly did nothing
for a listening socket. The vnode remained opened for connections. This
is fixed by removing vnode in uipc_close(). Maybe the right way would be
to do it for all sockets (not only listening), simply move the vnode
teardown from uipc_detach() to uipc_close()?
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D9770
It implements missing man(7) macros used in base by kerberos/ntp and makes them
supported by mandoc.
This import should have been done before the removal of groff.
Reported by: gordon
The cmd argument passed to extattrctl() is not decoded as a string constant
but is just printed in hex. The value is filesystem-specific but in
practice is only used with UFS1 filesystems.
All manpages in base are now compatible with mandoc(1), all roff documentation
will be relocated in the doc tree. man(1) can now use groff from the ports tree
if it needs.
Also remove checknr(1) and colcrt(1) which are only useful with groff.
Approved by: (no objections on the mailing lists)
- dup and dup2 print fd arguments in decimal.
- pread and pwrite are similar to read and write with the addition of the
file offset.
- getdirentries displays the output entries as a string for now and also
prints the value returned in *basep. Eventually the buffer for
getdirentries should perhaps be decoded as an array of dirent
structures.
PR: 214885
Submitted by: Jonathan de Boyne Pollard <J.deBoynePollard-newsgroups@NTLWorld.COM>
J. Sorenson and J. Webster, Strong pseudoprimes to twelve prime
bases, Math. Comp. 86(304):985-1003, 2017.
teach primes(6) to enumerate primes up to 2^64 - 1. Until Sorenson
and Webster's paper, we did not know how many strong speudoprime tests
were required when testing alleged primes between 3825123056546413051
and 2^64 - 1.
Reported by: Luiz Henrique de Figueiredo
Relnotes: primes(6) now enumerates primes beyond 3825123056546413050,
up to a new limit of 2^64 - 1.
MFC After: 1 week
Add a new sysdecode_getrusage_who() which decodes the RUSAGE_* constant
passed as the first argument to getrusage(). Use this function in both
kdump and truss to decode the first argument to getrusage().
PR: 215448
Submitted by: Anton Yuzhaninov <citrin+pr@citrin.ru>
MFC after: 1 month
This is undocumented to match GNU diff where -H is also undocumented.
Some existing software (such as kompare) uses this option by default.
Reviewed by: emaste, rpokala
Differential Revision: https://reviews.freebsd.org/D11022
^/head@r319125 changed the location of the backup pmbr, requiring the
output files to be regenerated, since they're binary disk dumps.
The output files were regenerated with "make rebase"--fixed in
^/head@r319294.
MFC with: r319125, r319294
PR: 219673
Sponsored by: Dell EMC Isilon
"make rebase" can be used for rebasing the output files from mkimg
after making a change to mkimg. This will come in handy soon, per
bug 219673.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
This helps ensure that the output files are regenerated if the input files
change, after the output files have been created.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
platform (returned by -m) can be different from the machine's processor
architecture (-p)
- Document that make(1) sets universal MACHINE and MACHINE_ARCH variables
based on these values
Reviewed by: imp, manpages (bjk)
Approved by: bjk, imp (implied)
Differential Revision: https://reviews.freebsd.org/D10489
The following changes have been made over the last couple of months:
Features:
- With bsdgrep -r, the working directory is implied if no directory is
specified
- bsdgrep will now behave as bsdgrep -r does when it's named rgrep
- bsdgrep now understands -z/--null-data to use \0 as EOL
- GNU regex compatibility is now indicated with a "GNU compatible" in
the version string
Fixes:
- --mmap no longer hangs when coming across an EOF without an
accompanying EOL
- -o/--color matching generally improved, now produces earliest /
longest matches
- Context output now more closely aligns with GNU grep
- Zero-length matches no longer exhibit broken behavior
- Every output line now honors -b/-H/-n flags
Tests have been added for previous regressions as well as other
previously untested behaviors.
Various other fixes have been commited, and refactoring for further /
later improvements has taken place.
(The original submission changed the version string to 2.5.2, but I
decided to use 2.6.0 to reflect the addition of new features.)
Submitted by: Kyle Evans <kevans91@ksu.edu>
Differential Revision: https://reviews.freebsd.org/D10982
The PMBR last sector should be number of sector - 1 (As stated in UEFI Spec
2.6 page 118 table 17).
This fixes warning printed by linux tools like parted or fdisk.
Sponsored by: Gandi.net
of calling grep -r without it, and saving the output to a file
This ensures that any errors thrown via grep -r are caught, not lost, and uses
existing atf-sh idioms for saving files.
Tested with: bsdgrep, gnu grep (base, ports)
Sponsored by: Dell EMC Isilon
- Add the missing option 'n' to the getopt(3) string
- Add the missing options 'libxo' and 'N' to the usage message
- Add the missing options 'M' and 'N' to the man-page
Submitted by: Keegan Drake H.P. <kdrakehp@zoho.com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D10915
Correct a couple of minor BSD grep assumptions that are valid for line
processing but not future chunk-based processing.
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: bapt, cem
Differential Revision: https://reviews.freebsd.org/D10824
import bsd diff3 from OpenBSD.
Differences with OpenBSD:
- lots of warning fixed
- no shell wrapper with diff3 actually living in libexec
- capsicumized
Keep it disconnected as it is not yet good enough to replace GNU diff
The motivation to import it now it to allow other people to jump in and also to
have an open development on it
Obtained from: OpenBSD
Decoding of the third argument depends on the first one. For doing this,
add a corresponding function to libsysdecode.
Thanks to jhb@ for suggesting this.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment. Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.
ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks. Unfortunately, not everything can be
fixed, especially outside the base system. For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.
Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.
Struct xvnode changed layout, no compat shims are provided.
For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.
Update note: strictly follow the instructions in UPDATING. Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.
Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver. Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).
Sponsored by: The FreeBSD Foundation (emaste, kib)
Differential revision: https://reviews.freebsd.org/D10439
catman(1) checks if mandoc(1) do support the manpage before trying to generate
the catpage and falls back on nroff, using the same mechanism as man(1).
Per POSIX, allow passing multiple pathnames to uncompress -c, concatenating
the uncompressed data.
Passing multiple pathnames to compress -c remains disallowed, since the
result cannot be decompressed.
PR: 219387
Reported by: Jörg Schilling
Metadata printing with -b, -H, or -n flags suffered from a few flaws:
1) -b/offset printing was broken when used in conjunction with -o
2) With -o, bsdgrep did not print metadata for every match/line, just
the first match of a line
3) There were no tests for this
Address these issues by outputting this data per-match if the -o flag is
specified, and prior to outputting any matches if -o but not --color,
since --color alone will not generate a new line of output for every
iteration over the matches.
To correct -b output, fudge the line offset as we're printing matches.
While here, make sure we're using grep_printline in -A context. Context
printing should *never* look at the parsing context, just the line.
The tests included do not pass with gnugrep in base due to it exhibiting
similar quirky behavior that bsdgrep previously exhibited.
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D10580
We should not set an arbitrary cap on the number of matches on a line,
and in any case MAX_LINE_MATCHES of 32 is much too low. Instead, if we
match more than MAX_LINE_MATCHES, keep processing and matching from the
last match until all are found.
For the regression test, we produce 4096 matches (larger than we expect
we'll ever set MAX_LINE_MATCHES) and make sure we actually get 4096
lines of output with the -o flag.
We'll also make sure that every distinct line is getting its own line
number to detect line metadata not being printed as appropriate along
the way.
PR: 218811
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reported by: jbeich
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D10577
r313948 partially fixed --mmap behavior but was incomplete. This commit
generally reverts it and does it the more correct way- by just consuming
the rest of the buffer and moving on.
PR: 219402
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D10820
This was done by Romain Tartière for PR123553. I initially thought that it would break code like this:
#define b00101010 -1
if (0 b00101010)
...
by joining 0 and b00101010 together. However, the real problem with that patch was that once it saw a 0, it assumed that the number was base 2, 8 or 16, ignoring base 10 floating point numbers. I fixed that.
I didn't copy the diagnostic part of the original patch as it seems out of scope of implementing binary integer literals formatting.
PR: 123553
Submitted by: romain (original version)
Approved by: pfg (mentor)
Items tested via this commit are:
- Some basic POSIX constants.
- Some valid programming environments with -v.
- Some invalid programming environments via -v.
NOTE: this test makes assumptions about ILP32/LP32 vs LP64 that are
currently not true on all architectures to avoid hardcoding some
architectures in the tests. I'm working on improving getconf(1) to be
more sane about handling ILP32/LP32 vs LP64. Future commits are coming
soon to address this.
MFC after: 2 weeks
Tested with: amd64, i386
Sponsored by: Dell EMC Isilon
These tests query a running process for information related to the -b,
-c, -e, and -f flags; the -f testcase is largely stubbed out, pending
additional work to determine a good, deterministic descriptor.
Core file test support is coming soon--it requires a bit more effort
due to the fact that:
- coredumps can be disabled (kern.coredump=0).
- corefiles can be put in different directories than the current
directory, or be named something other than `<prog>.core`
(`kern.corefile`).
MFC after: 2 months
Sponsored by: Dell EMC Isilon
Bring in some bits from NetBSD and lift the restriction in uniq(1) that
-c cannot be used with the -d and -u options. This restriction seems
unnecessary and is supported at least by GNU, OpenBSD, and NetBSD. Lift
the restriction and simplify the show() logic a little bit to maintain
functionality when -c is provided with -d/-u.
Also with this change, -d and -u are now actually a mutually exclusive,
albeit valid, combination. Given that they both indicate opposite
behavior, uniq(1) will no longer output anything if both -d and -u are
supplied. This is in line with NetBSD as well as GNU.
Adjust the man page and usage() to reflect that -c is its own standalone
option.
PR: 200553
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem, emaste
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D10694
Previously, when given a negative -A/-B/-C argument bsdgrep would
overflow the respective context flag(s) and exhibited surprising
behavior.
Fix this by removing unsignedness of Aflag/Bflag and erroring out if
we're given a value < 0. Also adjust the type used to track 'tail'
context in procfile() so that it accurately reflects the Aflag value
rather than overflowing and losing trailing context.
This also fixes an inconsistency previously existing between -n and
-C "n" behavior. They are now both limited to LLONG_MAX, to be
consistent.
Add some test cases to make sure grep errors out properly for both
negative context values as well as non-numeric context values rather
than giving bogus matches.
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D10675
The _Noreturn attribute was added to placate Coverity and other static
analysis tools. The __printflike attribute was added to catch issues
with the calls related to printf(3) abuse.
- Modify the code to facilitate the __printflike attribute addition.
- Convert errf calls in to_mb(..) and to_mb_string(..) to warn(..) so
the calls will return instead of exiting, as the code suggests it
should.
Differential Revision: D10704
MFC after: 1 month
Reviewed by: pfg
Sponsored by: Dell EMC Isilon
lib/libmt/mtlib.c:
Add the LTO-8 density code to the density table in libmt.
usr.bin/mt/mt.1:
Add the LTO-8 density code, tracks, bpmm, and bpi to the density
table in the mt(1) man page.
MFC after: 3 days
Sponsored by: Spectra Logic
- Fix a typo (SIGIGN -> SIG_IGN). Use .Dv when referencing SIG_IGN.
- Use semi-colons as soft breaks when separating sentences for
the FLAGS section.
- Tweak wording for C slightly to flow better and to be a bit
more technically correct (signals with handlers installed will
be caught by the target program).
- Reference signal(3) in the SEE ALSO section.
MFC after: 3 weeks
Sponsored by: Dell EMC Isilon
that setting the kernels' idea of terminal size is somehow an
alternative to environment variables.
Reported by: kib
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Add a -C option to specify a maximum capacity for the final image file.
It is useful to control the size of the generated image for sdcard or
when we will add dynamic size partition.
Add --capacity which is a shorthand to define min and max capacity at
the same time.
Reviewed by: bapt, marcel, wblock (manpages)
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D10509
When the input to csplit contains fewer lines than the number of matches
specified, extra output was mistakenly included in some output files.
Fix the bug and add a simple ATF regression test.
PR: 219024
Submitted by: J.R. Oldroyd <fbsd at opal.com>
size is already set to something other than zero. It's supposed to be
called from eg /etc/profile - it's not neccessary to query terminal
size when logging in over the network, because the protocol used already
takes care of this, but it's neccessary when logging over a serial line.
Reviewed by: cem, Daniel O'Connor <darius@dons.net.au>
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10637
for "unable to parse response" error which happens when youre typing
too fast for the machine you're running it on.
Reviewed by: cem, Daniel O'Connor <darius@dons.net.au>
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10624
Among new things it is now threaded by default, use zstd -T# to chose the
number of threads not that -T0 will automatically determine the number of
threads based on the number of CPU online.
This will help application developers simulate end of tape conditions.
To inject an error in sa0:
sysctl kern.cam.sa.0.inject_eom=1
This will return the next read or write request queued with 0 bytes
written. Any subsequent writes or reads will go along as usual.
This will also cause the early warning position flag to get set
for the next position query. So, 'mt status' will show the BPEW
(Beyond Programmable Early Warning) flag on the first query after
an error injection. After that, the position flags will be as they
are in the underlying tape drive.
Also, update the sa(4) man page to describe tape parameters,
which can be set via 'mt param'.
sys/cam/scsi/scsi_sa.c:
In saregister(), create the inject_eom sysctl variable.
In sastart(), check to see whether inject_eom is set. If
so, return the read or write with 0 bytes written to
indicate EOM. Set the set_pews_status flag so that we
fake PEWS status in the next position call for reads, and the
next 3 calls for writes. This allows the user to see the BPEW
flag one time via 'mt status'.
In sagetpos(), check the set_pews_status flag and fake
PEWS status and decrement the counter if it is set.
share/man/man4/sa.4:
Document the inject_eom sysctl variable.
Document all of the parameters currently supported via
'mt param'.
usr.bin/mt/mt.1:
Point the user to the sa(4) man page for more details on
supported parameters.
MFC after: 3 days
Sponsored by: Spectra Logic
Refactoring done in r317703 broke -c, -l, and -L flags implying
suppression of match printing. Fortunately this is just a matter of not
doing any printing of the resulting matches and context printing was not
broken in this refactoring.
Add some regression tests since this area may still see further
refactoring, include different context flags as well even though they
were not broken in this case.
PR: 219077
Submitted by: Kyle kevans91@ksu.edu
Reported by: markj
Reviewed by: cem, ngie
Differential Revision: https://reviews.freebsd.org/D10607
As of r295638, fputs() returns the number of bytes written (if not more than
INT_MAX). This broke csplit completely, since csplit assumed only success
only for the return value 0.
PR: 213510
Submitted by: J.R. Oldroyd
MFC after: 1 week
Relnotes: yes
In BSD grep, fix escape map building in the regex parser. It was
previously using memory not explicitly initialized, and the MBS escape
map was being built based on a version of the pattern with escapes
already parsed out.
This is Kyle's change, but I restored the broken style that already
exists in this file.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem, Kyle Evans (my style changes)
Differential Revision: https://reviews.freebsd.org/D10098
-w flag matching with an empty pattern was generally 'broken', allowing
matches to occur on any line whether or not it actually matches -w
criteria.
This fix required a good amount of refactoring to address. procline()
is altered to *only* process the line and return whether it was a match
or not, necessary to be able to short-circuit the whole function in case
of this matchall flag. -m flag handling is moved out as well because it
suffers from the same fate as context handling if we bypass any actual
pattern matching.
The matching context (matches, mostly) didn't previously exist outside
of procline(), so we go ahead and create context object for file
processing bits to pass around. grep_printline() was created due to
this, for the scenarios where the matches don't actually matter and we
just want to print a line or two, a la flushing the context queue and
no -o or --color specified.
Damage from this broken behavior would have been mitigated by the fact
that it is unlikely users would invoke grep -w with an empty pattern.
This was identified while checking PR 105221 for problems it this may
cause in BSD grep, but PR 105221 is *not* a report of this behavior.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Differential Revision: https://reviews.freebsd.org/D10433
r317049 added -z/--null-data to BSD grep but missed the update to nls
catalogs.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Differential Revision: https://reviews.freebsd.org/D10456
As reported in r218614 it's useful to have an indication of whether or not
BSD grep was built with GNU_GREP_COMPAT.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reported by: mandree
Differential Revision: https://reviews.freebsd.org/D10451
-w and -v flag matching was mostly functional but had some minor
problems:
1. -w flag processing only allowed one iteration through pattern
matching on a line. This was problematic if one pattern could match
more than once, or if there were multiple patterns and the earliest/
longest match was not the most ideal, and
2. Previous work "fixed" things to not further process a line if the
first iteration through patterns produced no matches. This is clearly
wrong if we're dealing with the more restrictive -w matching.
#2 breakage could have also occurred before recent broad rewrites, but
it would be more arbitrary based on input patterns as to whether or not
it actually affected things.
Fix both of these by forcing a retry of the patterns after advancing
just past the start of the first match if we're doing more restrictive
-w matching and we didn't get any hits to start with. Also move -v flag
processing outside of the loop so that we have a greater change to match
in the more restrictive cases. This wasn't strictly wrong, but it could
be a little more error prone.
While here, introduce some regressions tests for this behavior and fix
some excessive wrapping nearby that hindered readability. GNU grep
passes these new tests.
PR: 218467, 218811
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem, ngie
Differential Revision: https://reviews.freebsd.org/D10329
These are only built as part of the top-level 'build-tools' call for
'make buildworld'. They still need to be cleaned during the 'make clean'
treewalks though.
Reported by: markj
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Long ago, sh used to have its own optimized and restricted string formatting
implementation, which the printf builtin had to bypass via asprintf() to a
temporary buffer. Since sh has used libc's string formatting implementation
for a long time, remove the workaround.
Add a check to keep printf %c '' working the same way (output nothing);
POSIX allows both outputting nothing and outputting a NUL byte.
Also, this change avoids silently discarding format directives for whose
output asprintf() cannot allocate memory.
usr.bin/diff/diffreg.c: In function 'change':
usr.bin/diff/diffreg.c:1085: warning: 'i' may be used uninitialized in this function
This version of gcc is not smart enough to see that 'i' cannot actually
be used unitialized. However, the variable is confusingly re-used, so
it is better to give it another name, and clearly initialize it before
attempting to use it.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10484
in the file by adding the global (g) option at the end. Without it, only the
first match is replaced, subsequent ones are ignored. The intention of the
example is to demonstrate something else, but adding the g matches the example
to what the description says.
Discussed with: brd (on IRC)
MFC after: 1 week
Bugs have been found in the fastmatch implementation as used in bsdgrep.
Some have been fixed (r316495) while fixes for others are in review
(D10098).
In comparison with the fastmatch implementation, Kyle Evans found that:
- regex(3)'s performance with literal expressions offers a speed
improvement over fastmatch
- regex(3)'s performance, both with simple BREs and EREs, seems to be
comparable
The regex implementation was imported in r226035, and the commit message
reports:
This is a temporary solution until the whole regex library is
not replaced so that BSD grep development can continue and the
backported code gets some review and testing. This change only
improves scalability slightly, there is no big performance boost
yet but several minor bugs have been found and fixed.
Introduce a WITH_/WITHOUT_BSD_GREP_FASTMATCH knob to support testing
of both approaches.
PR: 175314, 194823
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: bdrewery (in part)
Differential Revision: https://reviews.freebsd.org/D10282
etcupdate(8) requires that option, while GNU diff supports many more variation
of that options, their behaviour beside the simple verion implemented here are
quite inconsistent as such I do not plan to implement those.
The only special keyword supported by this implementation are: %< and %>
%= is not implemented as the documentation of GNU diff says: common lines, but
it actually when tested print the changes from the first file
in place. To do per-cpu stats, convert all fields that previously were
maintained in the vmmeters that sit in pcpus to counter(9).
- Since some vmmeter stats may be touched at very early stages of boot,
before we have set up UMA and we can do counter_u64_alloc(), provide an
early counter mechanism:
o Leave one spare uint64_t in struct pcpu, named pc_early_dummy_counter.
o Point counter(9) fields of vmmeter to pcpu[0].pc_early_dummy_counter,
so that at early stages of boot, before counters are allocated we already
point to a counter that can be safely written to.
o For sparc64 that required a whole dummy pcpu[MAXCPU] array.
Further related changes:
- Don't include vmmeter.h into pcpu.h.
- vm.stats.vm.v_swappgsout and vm.stats.vm.v_swappgsin changed to 64-bit,
to match kernel representation.
- struct vmmeter hidden under _KERNEL, and only vmstat(1) is an exclusion.
This is based on benno@'s 4-year old patch:
https://lists.freebsd.org/pipermail/freebsd-arch/2013-July/014471.html
Reviewed by: kib, gallatin, marius, lidl
Differential Revision: https://reviews.freebsd.org/D10156
- Report missing includes at the correct location.
- Add initial support for the -@ option emitting a symbol table.
- Add support for running tests with and without -@
- Add support for generating __fixups__ and __local_fixups__
- Attach the to-string transform to the node path.
r316477 broke zero-length matches when not using the -o flag, by
skipping over them entirely.
Add a regression test so that it doesn't break again in the future.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem emaste ngie
Differential Revision: https://reviews.freebsd.org/D10333
Make bsdgrep more sensitive to context overlaps. If it's printing
context that either overlaps or is immediately adjacent to another bit
of context, don't print a separator.
- Non-overlapping segments no longer have two separators between them
- Overlapping segments no longer have separators between them with
overlapping sections repeated
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D10105
This is more sensible than the previous behaviour of grepping stdin,
and matches newer GNU grep behaviour.
PR: 216307
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem, emaste, ngie
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/
-z treats input and output data as sequences of lines terminated by a
zero byte instead of a newline. This brings it more in line with GNU grep
and brings us closer to passing the current tests with BSD grep.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: cem
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D10101
messages before accessing message fields that may not be present,
removing dead/duplicate/misleading code along the way.
Document the message format for each routing socket message in
route.h.
Fix a bug in usr.bin/netstat introduced in r287351 that resulted in
pointer computation with essentially random 16-bit offsets and
dereferencing of the results.
Reviewed by: ae
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D10330
The destination buffer is sized as the sum of program argument lengths, so
it has plenty of room for *argv. Appease Coverity by using strlcpy instead
of strcpy. Similar to a nearby cleanup performed in r316500.
No functional change.
Reported by: Coverity (CWE-120)
CID: 1006703
Sponsored by: Dell EMC Isilon
Create additional tests to cover regressions that were discovered by
PRs linked to reviews D10098, D10102, and D10104.
It is worth noting that neither bsdgrep(1) nor gnugrep(1) in the base
system currently pass all of these tests, and gnugrep(1) not quite being
up to snuff was also noted in at least one of the PRs.
PR: 175314 202022 195763 180990 197555 197531 181263 209116
Submitted by: Kyle Evans <kevans91@ksu.edu>
Reviewed by: cem, ngie, emaste
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D10112
Invalid expressions with an ultimate compiled pattern length of 0 (e.g.,
"grep -E {") were not taken into account and caused a segfault while trying
to fill in the good suffix table.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: me
Differential Revision: https://reviews.freebsd.org/D10113
xmalloc was a debug malloc implementation, but the x{malloc,calloc,free}
functions default to calling the malloc(3) equivalents.
Instead of relying on this malloc shim, we can devise better ways to debug
malloc issues that aren't misleading upon initial inspection. (I.e., using
jemalloc's various built-in debugging capabilities.)
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: emaste, cem
Differential Revision: https://reviews.freebsd.org/D10269
r316477 changed the color output to match exactly the in-tree GNU grep,
but introduces unnecessary escape sequences.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reported by: ache
MFC after: 1 month
MFC with: r316477
Create a convenience rgrep link for bsdgrep(1) that observes 'grep -r'
behavior.
A follow-up to r316473.
Submitted by: Kyle Evans <kevans91 at ksu.edu>
Reviewed by: emaste (earlier version), cem
Differential Revision: https://reviews.freebsd.org/D10109
- Set REG_NOTBOL if we've already matched beginning of line and we're
examining later parts
- For each pattern we examine, apply it to the remaining bits of the
line rather than (potentially) smaller subsets
- Check for REG_NOSUB after we've looked at all patterns initially
matching the line
- Keep track of the last match we made to later determine if we're
simply not matching any longer or if we need to proceed another byte
because we hit a zero-length match
- Match the earliest and longest bit of each line before moving the
beginning of what we match to further in the line, past the end of the
longest match; this generally matches how gnugrep(1) seems to behave,
and seems like pretty good behavior to me
- Finally, bail out of printing any matches if we were set to print all
(empty pattern) but -o (output matches) was set
PR: 195763, 180990, 197555, 197531, 181263, 209116
Submitted by: "Kyle Evans" <kevans91@ksu.edu>
Reviewed by: cem
MFC after: 1 month
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D10104
top(1) read the wrong amount of data from sysctl, uint64_t instead of
boolean_t, resulting in the stats not showing in many cases.
X-MFC-With: r315435
Sponsored by: ScaleEngine Inc.
resulting in a process dumping core in the corefile.
Also extend procstat to view select members of 'struct ptrace_lwpinfo'
from the contents of the note.
Sponsored by: Dell EMC Isilon
Show nanoseconds in the -u/-c header line.
The present portability conditionals cannot handle the POSIX standard
st_mtim, so remove them and unconditionally use st_mtim.
PR: 218018
Reported by: jbeich
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D10145
implemented in top(1), rather than relying on K&R prototypes, which can
cause problems on targets where there are multiple incompatible calling
conventions and the compiler requires argument information to select the
correct one.
(There's a bit more to do here, since it looks like top(1) also sometimes
provides prototypes for various curses functions rather than relying on
the header file...)
Sponsored by: DARPA, AFRL
MFC after: 1 week
Sync libarchive with vendor.
Vendor changes (FreeBSD-related):
- store extended attributes with extattr_set_link() if no fd is provided
- add extended attribute tests to libarchive and bsdtar
- fix tar's test_option_acls
- support the UF_HIDDEN file flag
X-MFC with: 315636
naming scheme
usr.bin/diff/diff_test was renamed to usr.bin/diff/netbsd_diff_test
to avoid collisions with the renamed FreeBSD test.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
This is a painful change, but it is needed. On the one hand, we avoid
modifying them, and this slows down some ideas, on the other hand we still
eventually modify them and tools like netstat(1) never work on next version of
FreeBSD. We maintain a ton of spares in them, and we already got some ifdef
hell at the end of tcpcb.
Details:
- Hide struct inpcb, struct tcpcb under _KERNEL || _WANT_FOO.
- Make struct xinpcb, struct xtcpcb pure API structures, not including
kernel structures inpcb and tcpcb inside. Export into these structures
the fields from inpcb and tcpcb that are known to be used, and put there
a ton of spare space.
- Make kernel and userland utilities compilable after these changes.
- Bump __FreeBSD_version.
Reviewed by: rrs, gnn
Differential Revision: D10018
Decode the last argument to ioctl() as a pointer rather than an int.
Eventually this could use 'int' for the _IOWINT() case and pointers for
all others.
The last argument to sendto() is a socklen_t value, not a pointer.
Provides:
amount of compressed data
logical size of compressed data (how much it would have taken uncompressed)
compression ratio (logical size : total ARC size)
Overhead (space consumed for compression headers)
Example output:
ARC: 31G Total, 18G MFU, 9067M MRU, 2236K Anon, 615M Header, 2947M Other
25G Compressed, 54G Uncompressed, 1.76:1 Ratio, 2265M Overhead
Reviewed by: jpaetzel, smh, imp, jhb (previous version)
MFC after: 2 week
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9829
Previously, the offset in a system call description specified the
array index of the start of a system call argument. For most system
call arguments this was the same as the index of the argument in the
function signature. 64-bit arguments (off_t and id_t values) passed
on 32-bit platforms use two slots in the array however. This was
handled by adding (QUAD_SLOTS - 1) to the slot indicies of any
subsequent arguments after a 64-bit argument (though written as ("{
Quad, 1 }, { Int, 1 + QUAD_SLOTS }" rather than "{ Quad, 1 }, { Int, 2
+ QUAD_SLOTS - 1 }"). If a system call contained multiple 64-bit
arguments (such as posix_fadvise()), then additional arguments would
need to use 'QUAD_SLOTS * 2' but remember to subtract 2 from the
initial number, etc. In addition, 32-bit powerpc requires 64-bit
arguments to be 64-bit aligned, so if the effective index in the array
of a 64-bit argument is odd, it needs QUAD_ALIGN added to the current
and any subsequent slots. However, if the effective index in the
array of a 64-bit argument was even, QUAD_ALIGN was omitted.
This approach was messy and error prone. This commit replaces it with
automated pre-processing of the system call table to do fixups for
64-bit argument offsets. The offset in a system call description now
indicates the index of an argument in the associated function call's
signature. A fixup function is run against each decoded system call
description during startup on 32-bit platforms. The fixup function
maintains an 'offset' value which holds an offset to be added to each
remaining system call argument's index. Initially offset is 0. When
a 64-bit system call argument is encountered, the offset is first
aligned to a 64-bit boundary (only on powerpc) and then incremented to
account for the second argument slot used by the argument. This
modified 'offset' is then applied to any remaining arguments. This
approach does require a few things that were not previously required:
1) Each system call description must now list arguments in ascending
order (existing ones all do) without using duplicate slots in the
register array. A new assert() should catch any future
descriptions which violate this rule.
2) A system call description is still permitted to omit arguments
(though none currently do), but if the call accepts 64-bit
arguments those cannot be omitted or incorrect results will be
displated on 32-bit systems.
Tested on: amd64 and i386
