Commit Graph

3562 Commits

Author SHA1 Message Date
Matt Macy
b52c534bff Add iflag=fullblock to dd
Normally, count=n means read(2) will be called n times on the input to dd. If
the read() returns short, as may happen when reading from a pipe, fewer bytes
will be copied from the input. With conv=sync the buffer is padded with zeros
to fill the rest of the block.

iflag=fullblock causes dd to continue reading until the block is full, so that
count=n means n full blocks are copied. This flag is compatible with illumos
and GNU dd and is used in the ZFS test suite.

Submitted by:	Ryan Moeller
Reviewed by:	manpages, mmacy@
MFC after:	1 week
Sponsored by:	 iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21441
2019-09-30 22:00:48 +00:00
Matt Macy
919156e34c Add oflag=fsync and oflag=sync capability to dd
Sets the O_FSYNC flag on the output file. oflag=fsync and oflag=sync are
synonyms just as O_FSYNC and O_SYNC are synonyms. This functionality is
intended to improve portability of dd commands in the ZFS test suite.

Submitted by:	Ryan Moeller
Reviewed by:	manpages, mmacy@
MFC after:	1 week
Sponsored by:	 iXsytems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21422
2019-09-30 21:56:42 +00:00
Matt Macy
2048fe7098 dd: Check result of close(2) for errors
close(2) can return errors from previous operations which should not be ignored.

PR: 229616
Submitted by:	Thomas Hurst
Reported by:	Thomas Hurst
Reviewed by:	mmacy@
Obtained from:	Ryan Moeller
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21376
2019-09-30 21:53:26 +00:00
Matt Macy
3b96efbda0 Add conv=fdatasync flag to dd
The fdatasync flag performs an fdatasync(2) on the output file before closing it.
This will be useful for the ZFS test suite.

Submitted by:	Ryan Moeller
Reviewed by:	manpages, mmacy@
MFC after:	1 week
Sponsored by:	iXSystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21373
2019-09-30 21:48:12 +00:00
Baptiste Daroussin
f370355791 Do not use our custom completion function, it is not needed anymore 2019-09-16 07:31:59 +00:00
Emmanuel Vadot
4c1a82cea5 pkgbase: Create a FreeBSD-utilities package and make it the default one
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21506
2019-09-05 14:15:47 +00:00
Matt Macy
ce1b19d8c8 Add conv=fsync flag to dd
The fsync flag performs an fsync(2) on the output file before closing it.
This will be useful for the ZFS test suite.

Submitted by:	ryan@ixsystems.com
Reviewed by:	jilles@, imp@
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-09-03 18:35:55 +00:00
Warner Losh
ab152d0263 Now that we have MK_LS_COLORS, we don't need RELEASE_CRUNCH check here.
The RELEASE_CRUNCH check is redundant here. We don't need it for releases
anymore, and picobsd can control this more directly without making it a special
case.
2019-07-15 07:35:46 +00:00
Bryan Drewery
c63c5ab001 Fix .depend files to work for build tools.
This is somewhat of a follow-up to r335746.

MFC after:	2 weeks
Sponsored by:	DellEMC
2019-06-15 17:08:13 +00:00
Stephen Hurd
705aad98c6 Some devices take undesired actions when RTS and DTR are
asserted. Some development boards for example will reset on DTR,
and some radio interfaces will transmit on RTS.

This patch allows "stty -f /dev/ttyu9.init -rtsdtr" to prevent
RTS and DTR from being asserted on open(), allowing these devices
to be used without problems.

Reviewed by:    imp
Differential Revision:  https://reviews.freebsd.org/D20031
2019-06-12 18:07:04 +00:00
Warner Losh
02e2acce1d Convert freebsd-version to using -v
newvers.sh has supported a variable setting only mode, use that in
preference to grep to future proof this script from changes there.

Differential Revision: https://reviews.freebsd.org/D19849
2019-05-23 17:19:00 +00:00
Warner Losh
8b82def341 Restore the -n flag parsing, but ignore it.
Since D19668 was done, new users of the -n flag have surfaced. Parse
and ignore it on the command line until they can be updated.

Suggested by: rgrimes (in D19668).
2019-04-24 05:24:10 +00:00
Edward Tomasz Napierala
fd5795b23a .Xr trim(8) from dd(1).
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2019-03-26 15:44:06 +00:00
Warner Losh
e77a99c1c1 Remove -n flag, fix setting date / time
r342139 bork setting the date. This fixes it by simply removing the -n
flag.

Differential Revision: https://reviews.freebsd.org/D19668
2019-03-21 06:47:23 +00:00
Warner Losh
ab87f59224 Remove the -d and -t flags from the man page
Remove -d and -t flags that were removed in r345050.

Noticed by: rgrimes@
2019-03-12 21:03:56 +00:00
Warner Losh
d24ba0bb86 Remove now useless -d and -t flags.
These were used to set dst flag and minutes west of UTC
respectively. These are obsolete and have been removed form the
kernel. These existed primarily to faithfully emulate early
Unix ABIs that have been removed from FreeBSD.

Reviewed by: jbh@, brooks@
Differential Revision: https://reviews.freebsd.org/D19550
2019-03-12 04:49:59 +00:00
Gordon Tetlow
244e58a2df Correct wording around '-' masks.
PR:		236407
Reported by:	Brian Saia
MFC after:	1 week
2019-03-09 00:30:13 +00:00
Jilles Tjoelker
56ada93f8a sh/tests: Improve failure messages of expansion/arith15.0 2019-03-07 22:51:58 +00:00
Jilles Tjoelker
484160a9cf sh: Add set -o pipefail
The pipefail option allows checking the exit status of all commands in a
pipeline more easily, at a limited cost of complexity in sh itself. It works
similarly to the option in bash, ksh93 and mksh.

Like ksh93 and unlike bash and mksh, the state of the option is saved when a
pipeline is started. Therefore, even in the case of commands like
  A | B &
a later change of the option does not change the exit status, the same way
  (A | B) &
works.

Since SIGPIPE is not handled specially, more work in the script is required
for a proper exit status for pipelines containing commands such as head that
may terminate successfully without reading all input. This can be something
like

(
        cmd1
        r=$?
        if [ "$r" -gt 128 ] && [ "$(kill -l "$r")" = PIPE ]; then
                exit 0
        else
                exit "$r"
        fi
) | head

PR:		224270
Relnotes:	yes
2019-02-24 21:05:13 +00:00
Jilles Tjoelker
94b793c497 sh: Send normal output from bind builtin to stdout
PR:		233343
Submitted by:	Yuichiro NAITO (original version)
2019-02-19 21:27:30 +00:00
Jilles Tjoelker
aac5464b61 sh: Restore $((x)) error checking after fix for $((-9223372036854775808))
SVN r342880 was designed to fix $((-9223372036854775808)) and things like
$((0x8000000000000000)) but also broke error detection for values of
variables without dollar sign ($((x))).

For compatibility, overflow in plain literals continues to be ignored and
the value is clamped to the boundary (except 9223372036854775808 which is
changed to -9223372036854775808).

Reviewed by:	se (although he would like error checking to be removed)
MFC after:	2 weeks
X-MFC-with:	r342880
Differential Revision:	https://reviews.freebsd.org/D18926
2019-02-10 22:23:05 +00:00
Edward Tomasz Napierala
719fd9fb2c Comment out the default sh(1) aliases for root, introduced in r343416.
The rest of this stuff is still to be discussed, but I think at this
point we have the agreement that the aliases should go.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2019-01-25 17:09:26 +00:00
Edward Tomasz Napierala
60315f8f9d Install .shrc for root, and set PS1 for the toor account.
Reviewed by:	jilles
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18872
2019-01-24 23:34:51 +00:00
Edward Tomasz Napierala
d81ca439e7 Make sh(1) support \u in PS1. This removes one fork/exec on interactive
shell startup.

Reviewed by:	0mp (man page), jilles
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18790
2019-01-24 11:59:46 +00:00
Edward Tomasz Napierala
ed1cfd30ce Don't mess with BLOCKSIZE in shell startup files - it's set by login.conf(5);
there's no need to even mention it in shell rc files.  Not that it's wrong;
just pointless and somewhat misleading.

Reviewed by:	jilles
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18809
2019-01-20 22:08:49 +00:00
Jilles Tjoelker
f91d2e2171 sh: Send libedit "ferr" output to fd 2
The libedit "fout" output must be sent to fd 2 since it contains prompts
that POSIX says must be sent to fd 2. However, the libedit "ferr" output
receives error messages such as from "bind" that make no sense to send to fd
1.
2019-01-20 14:25:25 +00:00
Dag-Erling Smørgrav
a96301b673 Fix an edge case when parsing large numbers which resulted in inconsistent
results between an expression that refers to a variable by name and the
same expression that includes the same variable by value.

Submitted by:	se@
MFC after:	1 week
2019-01-09 09:36:54 +00:00
Jilles Tjoelker
993b1e419c sh: Add test for exported but unset variables
PR:		233545
2019-01-03 20:23:12 +00:00
Jilles Tjoelker
56f33d07ce sh: Do not place exported but unset variables into the environment
PR:		233545
Submitted by:	Jan Beich
Obtained from:	NetBSD
2019-01-03 20:22:35 +00:00
Chris Rees
64e8790d7e Add a note that the use of -B option does not guarantee a size of fragment
if -z option also used.

Recommend the use of zip(1) if compressed files of predictable size needed.

PR:			docs/41089
Submitted by:		Sevan Janiyan
Reported by:		areilly@bigpond.net.au

While here, pet igor

Reviewed by:		bcr
Differential Revision:	https://reviews.freebsd.org/D18686
2018-12-29 23:08:59 +00:00
Edward Tomasz Napierala
20c9381c98 Make sh(1) collapse $HOME into "~" in PS1.
Reviewed by:	jilles
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18663
2018-12-28 17:51:40 +00:00
George V. Neville-Neil
53fc043d41 Remove, the now very outdated, timed.
Submitted by:	Kyle Spiers ksspiers at gmail
Reviewed by:	bcr,brooks,bz,sbruno
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D18505
2018-12-15 21:34:40 +00:00
Jilles Tjoelker
1becbc64f8 sh(1): Remove -c string from set builtin documentation
Altering the -c string at run time does not make sense and is not possible.

MFC after:	1 week
2018-12-08 12:49:19 +00:00
Sevan Janiyan
d7a570379b Clarify that /dev/kmem is not used.
Move cross ref to right place while here.

Submitted by:	kib
MFC after:	7 days
2018-12-03 20:01:51 +00:00
Sevan Janiyan
9588059256 Note these tools use kvm(3) and not procfs as in Solaris.
Obtained from:	NetBSD
MFC after:	7 days
2018-12-03 18:00:46 +00:00
Guangyuan Yang
e4c7371cef Clarify that patterns are extended regular expressions in pkill(1) manual page.
PR:		231060
Submitted by:	naddy
MFC after:	3 days
2018-12-01 05:58:33 +00:00
Maxim Sobolev
dead7b5e47 Replace hand-crafted naive byte-by-byte zero block detection routine
with macro based around memcmp(). The latter is expected to be some
8 times faster on a modern 64-bit architectures.

In practice, throughput of doing conv=sparse from /dev/zero to /dev/null
went up some 5-fold here from 1.9GB/sec to 9.7GB/sec with this change
(bs=128k).

MFC after:	2 weeks
2018-11-29 19:28:01 +00:00
Jilles Tjoelker
468ed39612 sh: Fix ${param?} default error message
If word in ${param?word} is missing, the shell shall write a default error
message. So expanding ${param?} when param is not set should write an error
message like

sh: param: parameter not set

This was broken by r316417.

PR:		233585
2018-11-28 20:03:53 +00:00
Jilles Tjoelker
77da4a95e8 sh: Use 126 and 127 exit status for failures opening a script
This affects scripts named on the command line, named with a '.' special
builtin and found via the PATH %func autoloading mechanism.

PR:		231986
2018-11-27 21:49:59 +00:00
Conrad Meyer
900c4ed3ca rm(1): Formalize non-functional status of -P flag
-P was introduced in 4.4BSD-Lite2 around 1994.  It overwrote file contents
with a pass of 0xff, 0x00, then 0xff, in a low effort attempt to "really
delete" files.

It has no user-visible effect; at the end of the day, the file is unlinked via
the filesystem.  Furthermore, the utility of overwriting files with patterned
data is extremely limited due to caveats at every layer of the stack[0] and
therefore mostly futile.  At the least, three passes is likely wasteful on
modern hardware[1].  It could also be seen as a violation of the "Unix
Philosophy" to do one thing per tiny, composable program.

Since 1994, FreeBSD has left it alone; OpenBSD replaced it with a single
pass of arc4random(3) output in 2012[2]; and NetBSD implemented partial, but
explicitly incomplete support for U.S. DoD 5220.22-M, "National Industrial
Security Program Operating Manual" in 2004[3].

NetBSD's enhanced comment above rm_overwrite makes a strong case for removing
the flag entirely:

> This is an expensive way to keep people from recovering files from your
> non-snapshotted FFS filesystems using fsdb(8).  Really.  No more.
>
> It is impossible to actually conform to the exact procedure given in
> [NISPOM] if one is overwriting a file, not an entire disk, because the
> procedure requires examination and comparison of the disk's defect lists.
> Any program that claims to securely erase *files* while conforming to the
> standard, then, is not correct.
>
> Furthermore, the presence of track caches, disk and controller write
> caches, and so forth make it extremely difficult to ensure that data have
> actually been written to the disk, particularly when one tries to repeatedly
> overwrite the same sectors in quick succession.  We call fsync(), but
> controllers with nonvolatile cache, as well as IDE disks that just plain lie
> about the stable storage of data, will defeat this.
>
> [NISPOM] requires physical media destruction, rather than any technique of
> the sort attempted here, for secret data.

As a first step towards evental removal, make it a placebo.  It's not like
it was serving any security function.  It is not defined in or mentioned by
POSIX.

If you are security conscious and need to erase your files, use a
woodchipper.  At a minimum, the entire disk needs to be overwritten, not
just one file.

[0]: https://www.ru.nl/publish/pages/909282/draft-paper.pdf
[1]: https://commons.erau.edu/cgi/viewcontent.cgi?article=1131&context=jdfsl
[2]: https://github.com/openbsd/src/commit/7c5c57ba81b5fe8ff2d4899ff643af18c
[3]: https://github.com/NetBSD/src/commit/fdf0a7a25e59af958fca1e2159921562cd

Reviewed by:	markj, Daniel O'Connor <darius AT dons.net.au> (previous version)
Differential Revision:	https://reviews.freebsd.org/D17906
2018-11-10 20:26:55 +00:00
Jilles Tjoelker
bb324af62a sh: Unify EXERROR and EXEXEC
The difference between EXERROR and EXEXEC was that EXEXEC passed along
exitstatus and EXERROR set exitstatus to 2 in the handling code.

By changing the places that raised EXERROR to set exitstatus to 2, the
handling of EXERROR and EXEXEC becomes the same.
2018-11-09 14:58:24 +00:00
Mariusz Zaborski
377421df96 capsicum: use a new capsicum helpers in tools
Use caph_{rights,ioctls,fcntls}_limit to simplify the code.
2018-11-04 19:24:49 +00:00
Conrad Meyer
d83db3fb6a Drop ed(1) "crypto"
You should not be using DES.  You should not have been using DES for the
past 30 years.

The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself.  In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).

Some general approaches ed(1) users might consider to replace the removed
DES mode:

1. Full disk encryption with something like AES-XTS.  This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest.  Like CBC, it lacks tampering protection.  Examples include
GELI, LUKS, FileVault2.

2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).

3. Native encryption at the filesystem layer.  Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.

4. Storing your files unencrypted.  It's not like DES was doing you much
good.

If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:

  openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>

Reviewed by:	allanjude, bapt, emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D17829
2018-11-04 17:56:16 +00:00
Mark Johnston
49618eacb2 Avoid copying a struct stat for acl_from_stat() calls.
CID:		1375584
MFC after:	2 weeks
2018-11-01 17:45:29 +00:00
Mateusz Piotrowski
f11e51a920 ps(1): Pet mandoc and igor
- Use Xr to reference other manual pages.
- Reference execve(2) instead of exec(2) as exec(2) does not exist.
- Remove the deprecated "Tn" macro.
- Improve the formatting of the etime description.

Reviewed by:	bcr
Approved by:	krion (mentor, implicit), mat (mentor, implicit)
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17780
2018-10-31 17:47:08 +00:00
Mateusz Piotrowski
c97c08de8a ps(1): Add a standard exit status section
Reviewed by:	bcr, eadler
Approved by:	krion (mentor)
MFC after:      3 days
Differential Revision:	https://reviews.freebsd.org/D17146
2018-10-31 16:10:04 +00:00
Jilles Tjoelker
b5532964e7 sh: Use exitstatus instead of exerrno to pass EXEXEC status
No functional change is intended.
2018-10-27 20:17:57 +00:00
Mark Johnston
15e594351a Don't set NFSv4 ACL inheritance flags on non-directories.
They only make sense in the context of directory ACLs, and attempting
to set them on regular files results in errors, causing a recursive
setfacl invocation to abort.

This is derived from patches by Shawn Webb <shawn.webb@hardenedbsd.org>
and Mitchell Horne <mhorne063@gmail.com>.

PR:		155163
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15061
2018-10-26 21:17:06 +00:00
Mark Johnston
10cb5344fd Don't print pathconf() errors if the target file doesn't exist.
The subsequent acl_get_file(3) call will simply echo the same error.

PR:		229930
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2018-10-26 19:01:52 +00:00
Mark Johnston
f024dd3bf0 Avoid leaking memory in error paths.
CID:		1390906
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
2018-10-26 18:56:58 +00:00