Sets the O_FSYNC flag on the output file. oflag=fsync and oflag=sync are
synonyms just as O_FSYNC and O_SYNC are synonyms. This functionality is
intended to improve portability of dd commands in the ZFS test suite.
Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXsytems, Inc.
Differential Revision: https://reviews.freebsd.org/D21422
close(2) can return errors from previous operations which should not be ignored.
PR: 229616
Submitted by: Thomas Hurst
Reported by: Thomas Hurst
Reviewed by: mmacy@
Obtained from: Ryan Moeller
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D21376
The fdatasync flag performs an fdatasync(2) on the output file before closing it.
This will be useful for the ZFS test suite.
Submitted by: Ryan Moeller
Reviewed by: manpages, mmacy@
MFC after: 1 week
Sponsored by: iXSystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21373
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by: bapt, gjb
Differential Revision: https://reviews.freebsd.org/D21506
The fsync flag performs an fsync(2) on the output file before closing it.
This will be useful for the ZFS test suite.
Submitted by: ryan@ixsystems.com
Reviewed by: jilles@, imp@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
The RELEASE_CRUNCH check is redundant here. We don't need it for releases
anymore, and picobsd can control this more directly without making it a special
case.
asserted. Some development boards for example will reset on DTR,
and some radio interfaces will transmit on RTS.
This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent
RTS and DTR from being asserted on open(), allowing these devices
to be used without problems.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D20031
newvers.sh has supported a variable setting only mode, use that in
preference to grep to future proof this script from changes there.
Differential Revision: https://reviews.freebsd.org/D19849
Since D19668 was done, new users of the -n flag have surfaced. Parse
and ignore it on the command line until they can be updated.
Suggested by: rgrimes (in D19668).
These were used to set dst flag and minutes west of UTC
respectively. These are obsolete and have been removed form the
kernel. These existed primarily to faithfully emulate early
Unix ABIs that have been removed from FreeBSD.
Reviewed by: jbh@, brooks@
Differential Revision: https://reviews.freebsd.org/D19550
The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option in bash, ksh93 and mksh.
Like ksh93 and unlike bash and mksh, the state of the option is saved when a
pipeline is started. Therefore, even in the case of commands like
A | B &
a later change of the option does not change the exit status, the same way
(A | B) &
works.
Since SIGPIPE is not handled specially, more work in the script is required
for a proper exit status for pipelines containing commands such as head that
may terminate successfully without reading all input. This can be something
like
(
cmd1
r=$?
if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then
exit 0
else
exit "$r"
fi
) | head
PR: 224270
Relnotes: yes
SVN r342880 was designed to fix $((-9223372036854775808)) and things like
$((0x8000000000000000)) but also broke error detection for values of
variables without dollar sign ($((x))).
For compatibility, overflow in plain literals continues to be ignored and
the value is clamped to the boundary (except 9223372036854775808 which is
changed to -9223372036854775808).
Reviewed by: se (although he would like error checking to be removed)
MFC after: 2 weeks
X-MFC-with: r342880
Differential Revision: https://reviews.freebsd.org/D18926
The rest of this stuff is still to be discussed, but I think at this
point we have the agreement that the aliases should go.
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
there's no need to even mention it in shell rc files. Not that it's wrong;
just pointless and somewhat misleading.
Reviewed by: jilles
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D18809
The libedit "fout" output must be sent to fd 2 since it contains prompts
that POSIX says must be sent to fd 2. However, the libedit "ferr" output
receives error messages such as from "bind" that make no sense to send to fd
1.
results between an expression that refers to a variable by name and the
same expression that includes the same variable by value.
Submitted by: se@
MFC after: 1 week
if -z option also used.
Recommend the use of zip(1) if compressed files of predictable size needed.
PR: docs/41089
Submitted by: Sevan Janiyan
Reported by: areilly@bigpond.net.au
While here, pet igor
Reviewed by: bcr
Differential Revision: https://reviews.freebsd.org/D18686
with macro based around memcmp(). The latter is expected to be some
8 times faster on a modern 64-bit architectures.
In practice, throughput of doing conv=sparse from /dev/zero to /dev/null
went up some 5-fold here from 1.9GB/sec to 9.7GB/sec with this change
(bs=128k).
MFC after: 2 weeks
If word in ${param?word} is missing, the shell shall write a default error
message. So expanding ${param?} when param is not set should write an error
message like
sh: param: parameter not set
This was broken by r316417.
PR: 233585
-P was introduced in 4.4BSD-Lite2 around 1994. It overwrote file contents
with a pass of 0xff, 0x00, then 0xff, in a low effort attempt to "really
delete" files.
It has no user-visible effect; at the end of the day, the file is unlinked via
the filesystem. Furthermore, the utility of overwriting files with patterned
data is extremely limited due to caveats at every layer of the stack[0] and
therefore mostly futile. At the least, three passes is likely wasteful on
modern hardware[1]. It could also be seen as a violation of the "Unix
Philosophy" to do one thing per tiny, composable program.
Since 1994, FreeBSD has left it alone; OpenBSD replaced it with a single
pass of arc4random(3) output in 2012[2]; and NetBSD implemented partial, but
explicitly incomplete support for U.S. DoD 5220.22-M, "National Industrial
Security Program Operating Manual" in 2004[3].
NetBSD's enhanced comment above rm_overwrite makes a strong case for removing
the flag entirely:
> This is an expensive way to keep people from recovering files from your
> non-snapshotted FFS filesystems using fsdb(8). Really. No more.
>
> It is impossible to actually conform to the exact procedure given in
> [NISPOM] if one is overwriting a file, not an entire disk, because the
> procedure requires examination and comparison of the disk's defect lists.
> Any program that claims to securely erase *files* while conforming to the
> standard, then, is not correct.
>
> Furthermore, the presence of track caches, disk and controller write
> caches, and so forth make it extremely difficult to ensure that data have
> actually been written to the disk, particularly when one tries to repeatedly
> overwrite the same sectors in quick succession. We call fsync(), but
> controllers with nonvolatile cache, as well as IDE disks that just plain lie
> about the stable storage of data, will defeat this.
>
> [NISPOM] requires physical media destruction, rather than any technique of
> the sort attempted here, for secret data.
As a first step towards evental removal, make it a placebo. It's not like
it was serving any security function. It is not defined in or mentioned by
POSIX.
If you are security conscious and need to erase your files, use a
woodchipper. At a minimum, the entire disk needs to be overwritten, not
just one file.
[0]: https://www.ru.nl/publish/pages/909282/draft-paper.pdf
[1]: https://commons.erau.edu/cgi/viewcontent.cgi?article=1131&context=jdfsl
[2]: https://github.com/openbsd/src/commit/7c5c57ba81b5fe8ff2d4899ff643af18c
[3]: https://github.com/NetBSD/src/commit/fdf0a7a25e59af958fca1e2159921562cd
Reviewed by: markj, Daniel O'Connor <darius AT dons.net.au> (previous version)
Differential Revision: https://reviews.freebsd.org/D17906
The difference between EXERROR and EXEXEC was that EXEXEC passed along
exitstatus and EXERROR set exitstatus to 2 in the handling code.
By changing the places that raised EXERROR to set exitstatus to 2, the
handling of EXERROR and EXEXEC becomes the same.
You should not be using DES. You should not have been using DES for the
past 30 years.
The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself. In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).
Some general approaches ed(1) users might consider to replace the removed
DES mode:
1. Full disk encryption with something like AES-XTS. This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest. Like CBC, it lacks tampering protection. Examples include
GELI, LUKS, FileVault2.
2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).
3. Native encryption at the filesystem layer. Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.
4. Storing your files unencrypted. It's not like DES was doing you much
good.
If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:
openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>
Reviewed by: allanjude, bapt, emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D17829
- Use Xr to reference other manual pages.
- Reference execve(2) instead of exec(2) as exec(2) does not exist.
- Remove the deprecated "Tn" macro.
- Improve the formatting of the etime description.
Reviewed by: bcr
Approved by: krion (mentor, implicit), mat (mentor, implicit)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D17780
They only make sense in the context of directory ACLs, and attempting
to set them on regular files results in errors, causing a recursive
setfacl invocation to abort.
This is derived from patches by Shawn Webb <shawn.webb@hardenedbsd.org>
and Mitchell Horne <mhorne063@gmail.com>.
PR: 155163
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D15061
Output padding is specified via outlen, which is set using the return value
of fprintf. Because it's printing that padding plus a trailing byte, it
grows by one each iteration rather than reflecting actual length.
Additionally, iec was sized improperly for scaling up similarly to si.
Fixing this revealed that the humanize_number(3) call to populate persec
was using the wrong width.
Submitted by: Thomas Hurst <tom@hur.st>
Reviewed by: imp
Approved by: re (kib)
Differential Revision: https://reviews.freebsd.org/D16960
Restore the original behavior of unlink(1), passing the provided filename
directly to unlink(2), handling the first argument being "--" correctly.
This fixes "unlink -foo", broken in r97533.
PR: 228448
Submitted by: Brennan Vincent <brennan@umanwizard.com> (original version)
Submitted by: Yuri Pankov
Reported by: Brennan Vincent <brennan@umanwizard.com>
Reviewed by: emaste, kevans, vangyzen, 0mp
Approved by: re (delphij)
Differential Revision: https://reviews.freebsd.org/D17132
The intention is to lower the value of the pointer, which according to ubsan
cannot be done by adding an unsigned quantity.
Reported by: kevans
Approved by: re (kib)
MFC after: 1 week
These aliases are supported and documented in the man page. For now, they
will not be mentioned in the error when an invalid argument is encountered,
instead keeping that list to the shorter 'preferred' names of each argument.
Reported by: rgrimes
--color may be set to one of: 'auto', 'always', and 'never'.
'auto' is the default behavior- output colors only if -G or COLORTERM are
set, and only if stdout is a tty.
'always' is a new behavior- output colors always. termcap(5) will be
consulted unless TERM is unset or not a recognized terminal, in which case
ls(1) will fall back to explicitly outputting ANSI escape sequences.
'never' to turn off any environment variable and -G usage.
Reviewed by: cem, 0mp (both modulo last-minute manpage changes
Differential Revision: https://reviews.freebsd.org/D16741
CLICOLOR will behavior as always- if present at all in the environment,
allow colors.
COLORTERM, recently enforced, will have to be both present and not empty.
Submitted by: imp
This fixes the build and I will redo these changes as part of a future review
that organizes them differently. The way I tried to do it here could be done
better. Sorry for the noise.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16737
Notable changes from what landed in r337505:
- sigalarm handler isn't setup unless we're actually using it
- Humanized versions of the amount of data transferred in the progress
update
Submitted by: imp
Reviewed by: kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D16642
This moves the symlink creation to after where the files are installed.
This also inverts the shell change so that it only happens if MK_TCSH is on.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16725
This simplifies pkgbase by migrating these to CONFS so they are properly
tagged as config files.
Approved by: will (mentor)
Differential Revision: https://reviews.freebsd.org/D16708
COLORTERM is the de facto standard, while CLICOLOR is generally specific to
FreeBSD and ls(1).
PR: 230101
Submitted by: D Green <dfrg@xsmail.com> (with manpage additions by myself)
Reviewed by: cem ("LGTM" in PR; pre-manpage changes)
MFC after: 1 week
This reports the current status on a single line every second, mirroring
similar functionality in GNU dd, and carefully interacts with SIGINFO.
PR: 229615
Submitted by: Thomas Hurst <tom@hur.st> (modified for style(9) nits by me)
MFC after: 1 week
Using padvance() requires undoing its append of '/' and prevents adjusting
its '%' logic to allow most directories with '%' in PATH.
No functional change is intended.
These are not used to link the final tool anymore. At some point in the past
the suffix rules changed to not link these in. The original reason for this in
r19176 is unclear but seems to be related to mkdep. The .depend handling is
still broken here as it is for all build tool patterns like this.
Sponsored by: Dell EMC
POSIX requires accepting unquoted newlines in word in parameter expansions
like ${param+word}, ${param#word}, although the Bourne shell did not support
it, it is not commonly used and might make it harder to find a missing
closing brace.
It was also strange that something like
foo="${bar#
}"
was rejected.
Reported by: Martijn Dekker via Robert Elz
Add a -R option to setfacl to operate recursively on directories, along
with the accompanying flags -H, -L, and -P (whose behaviour mimics
chmod).
A patch was submitted with PR 155163, but this is a new implementation
based on comments raised in the Phabricator review for that patch
(review D9096).
PR: 155163
Submitted by: Mitchell Horne <mhorne063@gmail.com>
Reviewed by: jilles
MFC after: 2 weeks
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14934
Apply patch submitted with PR 217159 to make ps use unlimited
width when not associated with a terminal (i.e., none of stdout, stdin,
or stderr is a tty). Update comments and man page correspondingly.
This change was requested to work around lack of -ww in scripts from
third-party packages, including Hadoop, and adds a small measure of
Linux compatibility. Hopefully few if any non-interactive scripts
depend on the old default of 79.
PR: 217159
Submitted by: n.deepak at gmail.com
Reviewed by: vangyzen jhb
Differential Revision: https://reviews.freebsd.org/D14614
Revert r314685, and add a comment describing the original
behavior and the intent.
Reviewed by: dab@ vangyzen@ jhb@
Differential Revision: https://reviews.freebsd.org/D14530
to see how much space it on them.
Adjust MOUNT_CHAR_DEVS to allow the free space of already mounted
devices to be displayed and report an appropriate error if the
device isn't mounted.
Reviewed by: cem
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D8801
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.
Reviewed By: bdrewery
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
Use a setup_sig() helper and make it fail when either of sigaction fails.
While there, do not leak fds for "." + minor cleanup.
Obtained from: OpenBSD (through DragonFly git eca362d0f9bd086cc56d6b5bc4f03f09e040b9db)
- Simplify the description of -H to assume 1:1 threading.
- Drop 'process' from description of 'lwp' field and the corresponding
XO field name.
- Do add an expansion of LWP in the description of 'lwp' and 'nlwps'.
- Add 'tid' as an alias for the 'lwp' field.
Reviewed by: imp, kib (older version)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D14021
libxo imposes a large burden on system utilities. In the case of ls, that
burden is difficult to justify -- any language that can interact with json
output can use readdir(3) and stat(2).
Logically, this reverts r291607, r285857, r285803, r285734, r285425,
r284494, r284489, r284252, and r284198.
Kyua tests continue to pass (libxo integration was entirely untested).
Reported by: many
Reviewed by: imp
Discussed with: manu, bdrewery
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13959
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
The wrong index was being checked for == ' ' in the resulting stringified
mode from strmode(3) -- it should have been the 11th value, not the 10th.
MFC after: 3 days
PR: 76711
Submitted by: Vasil Dimov <vd@datamax.bg>
Previously it just didn't work at all - kvm_getprocs(3) doesn't update
the &nentries when it returns NULL. The end result was that ps(1) showed
garbage data instead of reporting kinfo_proc size mismatch.
Reviewed by: cem
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12414
If they are still needed, you can find them in the net/bsdrcmds port.
This was proposed June, 20th and approved by various committers [1].
They have been marked as deprecated on CURRENT in r320644 [2] on July, 4th.
Both stable/11 and release/11.1 contain the deprecation notice (thanks to
allanjude@).
Note that ruptime(1)/rwho(1)/rwhod(8) were initially thought to be part of
rcmds but this was a mistake and those are therefore NOT removed.
[1] https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html
[2] https://svnweb.freebsd.org/base?view=revision&revision=320644
Reviewed by: bapt, brooks
Differential Revision: https://reviews.freebsd.org/D12573
from blank lines being numbered or unnumbered, depending on whether cat
was invoked with -ne or -be.
At present, when cat is invoked with -be, there is an aditional
difference that the '$' on blank lines is placed on the far left of the
output.
Discussed in bug 210607.
While here, revert the workaround from r304035 which skipped the unit test for
this issue previously.
PR: 210607
Submitted by: myself
Reviewed by: bdrewery
Obtained from: NetBSD
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D12432
Add basic command line parsing test coverage for these utilities. The tests
were automatically generated based on their man pages. These tests can be
expanded by hand for more thorough coverage. The aim is to generate very
basic amount of test coverage for all the utilities in the base system.
Submitted by: shivansh
Reviewed by: asomers, brooks
MFC after: 3 weeks
Sponsored by: Google, Inc (GSoC 2017)
Differential Revision: https://reviews.freebsd.org/D12036
we do for the usual nice values. It could be argued that they should
use another set of indicators, since the underlying mechanism is
different, but they match the description in the manual page, and so
I think it's ok to not overcomplicate things.
PR: 81757
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Though technically correct, GCC complains about usingi a "%zd" format
specifier for a long.
Reported by: cem
MFC after: 3 weeks
X-MFC-With: 322893
Sponsored by: Spectra Logic
dd(1) casts many of its numeric arguments from uintmax_t to intmax_t and
back again to detect whether or not the original arguments were negative.
This is not correct, and causes problems with boundary cases, for example
when count is SSIZE_MAX-1.
PR: 191263
Submitted by: will@worrbase.com
Reviewed by: pi, asomers
MFC after: 3 weeks
output to not include a trailing new line, which is a potential POLA violation
for existing consumers. Change pgrep to always emit a trailing new line on
completion of its output, regardless of the delimeter in use (which technically
is also a potential POLA violation for existing consumers that rely on the
pre-r322210 buggy behaviour, but a line has to be drawn somewhere).
PR: 221534
Submitted by: kdrakehp zoho com
Reported by: kdrakehp zoho com
MFC after: 1 week
X-MFC-with: r322210
Non-tests/... changes:
- Add HAS_TESTS= to Makefiles with libraries and programs to enable iteration
and propagate the appropriate environment down to *.test.mk.
tests/... changes:
- Add appropriate support Makefile.inc's to set HAS_TESTS in a minimal manner,
since tests/... is a special subdirectory tree compared to the others.
MFC after: 2 months
MFC with: r322511
Reviewed by: arch (silence), testing (silence)
Differential Revision: D12014
e.g. "pgrep -d, getty" outputs "1399,1386,1309,1308,1307,1306,1305,1302,"
Ensure the list is correctly delimited by suppressing the emission of the
delimiter after the final PID.
Reviewed by: imp, kib
MFC after: 1 week
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D8537
This will prevent false positives from occurring if the test is run on
ZFS since ZFS doesn't support fflags throbbing like UFS.
PR: 221189
MFC after: 4 days
MFC with: r321949
- :f_flag fails on ZFS because UF_IMMUTABLE isn't supported [1].
- :v_flag fails on ZFS because the mode for foo is [always] updated
unnecessarily.
get_filesystem(..) (supporting function that was added to the test script)
is based on equivalent logic in usr.bin/extattr/tests/extattr_test.sh .
MFC after: 1 week
PR: 221189 [1], 221188 [2]
directories to SUBDIR.${MK_TESTS} idiom
This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .
No functional change intended.
MFC after: 1 weeks
localtime(3) returns NULL when passed an invalid time_t but date(1)
previously did not handle it. Exit with an error in that case.
PR: 220828
Reported by: Vinícius Zavam
Reviewed by: cem, kevans
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D11660
Apart from the fact that subtle syntactic changes make a poor compile-time
option, the NOHACK case has been obviously broken since it was added,
because it uses q uninitialized if (*p != '\0').
No functional change is intended.
After the addition of SUBDIR.yes, uniquifying/ordering the SUBDIRs doesn't
make a whole lot of sense, and it's in effect a half measure.
Ordering SUBDIR (after adding SUBDIR.yes to it) in bsd.subdir.mk is a
separate change that warrants more discussion/testing, because while
the SUBDIR_PARALLEL work largely fixed dependency ordering for SUBDIRs,
there might be downstream FreeBSD consumers that rely on the SUBDIR
ordering.
MFC after: 2 months
Reviewed by: bdrewery
Differential Revision: D11398
