Commit Graph

186 Commits

Author SHA1 Message Date
Alex Richardson
6e4162c60c install(1): Avoid unncessary fstatfs() calls and use mmap() based on size
According to git blame the trymmap() function was added in 1996 to skip
mmap() calls for NFS file systems. However, nowadays mmap() should be
perfectly safe even on NFS. Importantly, onl ufs and cd9660 file systems
were whitelisted so we don't use mmap() on ZFS. It also prevents the use
of mmap() when bootstrapping from macOS/Linux since on those systems the
trymmap() function was always returning zero due to the missing MFSNAMELEN
define.

This change keeps the trymmap() function but changes it to check whether
using mmap() can reduce the number of system calls that are required.
Using mmap() only reduces the number of system calls if we need multiple read()
syscalls, i.e. if the file size is > MAXBSIZE. However, mmap() is more expensive
than read() so this sets the threshold at 4 fewer syscalls. Additionally, for
larger file size mmap() can significantly increase the number of page faults,
so avoid it in that case.

It's unclear whether using mmap() is ever faster than a read with an appropriate
buffer size, but this change at least removes two unnecessary system calls
for every file that is installed.

Reviewed By:	markj
Differential Revision: https://reviews.freebsd.org/D26041
2020-10-14 12:28:41 +00:00
Alex Richardson
0fd54da0b0 Allow install(1)'s create_tempfile() to work on Linux hosts
GLibc expects six 'X' characters in the mkstemp template argument and
will return EINVAL otherwise.

Reviewed By:	emaste, imp, mjg
Differential Revision: https://reviews.freebsd.org/D25662
2020-07-15 17:24:39 +00:00
Eugene Grosbein
57f09d08c6 install(1): another correction after r363064
Make sure we call fsync(2) on strip result
in case of "safecopy" and "strip -o tempcopy -- src"
before renaming tempcopy to destination.

MFC after:	3 weeks
X-MFC-With:	r363064
2020-07-12 20:59:52 +00:00
Eugene Grosbein
a0b083fbda Fix the tests for install(1): add support for STRIPBIN's -o option.
Reported by:	lwhsu
MFC after:	1 week
2020-07-10 22:59:26 +00:00
Eugene Grosbein
cd88eaa5b4 install(1): addition to r363067
Also, make it not break if STRIPBIN points to strip version without -o support
and destination does not exist before installing.

Reported by:	lwhsu
MFC after:      1 month
X-MFC-With:     363064
2020-07-10 17:42:22 +00:00
Eugene Grosbein
8ce741ca77 install(1): correction after r363064
Make it not break if STRIPBIN points to strip version without -o support.
In that case, perform extra copy just like before r363064.

MFC after:	1 month
X-MFC-With:	363064
2020-07-10 00:45:34 +00:00
Eugene Grosbein
2d68fac9bd Optimize install(1) a bit.
Currently, "install -s -S" behaviour is inefficient for upgrade.
First it finds that destination file already exists and copies
source file to temporary file. Then it calls strip(1)
with name of temporary file as single agrument and our strip(1) creates
another temporary file in the /tmp (or TMPDIR) making another copy
that is finally copied to DESTDIR third time.

Meantime, strip(1) has an option "-o dst" to specify destination
so install(1) is allowed to skip initial copying from obj to DESTDIR.
This change makes it do so.

Take a look at https://reviews.freebsd.org/D25551 for details
and efficiency numbers (in short: upto 32% gained for installword).

MFC after:		1 month
Differential Revision:	https://reviews.freebsd.org/D25551
2020-07-10 00:24:42 +00:00
Alex Richardson
52918a7b1b Allow boostrapping xinstall on Linux
Linux does not have st_flags so we have to avoid using it there.

Reviewed By:	emaste, imp
Differential Revision: https://reviews.freebsd.org/D22446
2019-11-20 17:24:49 +00:00
Mateusz Piotrowski
9236a881f6 install.1: Add missing arguments to option descriptions
While here:
- Remove deprecated Tn macros.
- Do not use macros with the -width option.

Reviewed by:	bcr
Approved by:	doc (bcr)
Differential Revision:	https://reviews.freebsd.org/D21173
2019-08-12 10:57:56 +00:00
Kyle Evans
6cbda6d943 install(1): Fix relative path calculation with partial common dest/src
For example, from the referenced PR [1]:

$ mkdir /tmp/lib/ /tmp/libexec
$ touch /tmp/lib/foo.so
$ install -lrs /tmp/lib/foo.so /tmp/libexec/

The common path identification bits terminate src at /tmp/lib/ and the
destination at /tmp/libe. The subsequent backtracking is then incorrect, as
it traverses the destination and backtraces exactly one level while eating
the 'libexec' because it was previously (falsely) identified as common with
'lib'.

The obvious fix would be to make sure we've actually terminated just after
directory separators and rewind a character if we haven't. In the above
example, we would end up rewinding to /tmp/ and subsequently doing the right
thing.

Test case added.

PR:		235330 [1]
MFC after:	1 week
2019-01-31 05:20:11 +00:00
Kirk McKusick
d81347652c When doing a -S "safe copy", the install command should do an
fsync(2) system call after copying the installed file to ensure
that it is on stable storage.

PR:          230851
Reviewed by: kib
Approved by: re (marius)
2018-08-27 15:20:42 +00:00
Alex Richardson
5f40118235 Allow xinstall and makefs to be crossbuilt on Linux and Mac
I need these tools in order to install the crossbuilt FreeBSD and create a
disk image. Linux does not have a st_flags in struct stat so unfortunately
I need a bunch of ugly ifdefs. The resulting binaries allow me to
sucessfully install a MIPS64 world and create a disk-image that boots.

Reviewed By:	brooks, bdrewery, emaste
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13307
2018-01-16 21:43:46 +00:00
Pedro F. Giffuni
8a16b7a18f General further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:49:47 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Mateusz Guzik
92b25dcd61 xinstall: plug an infinite loop in directory creation
If stat continues to fail with ENOENT and mkdir with EEXIST the code wont
finish. In particular this can show up when the target path follows through
a symlink to a non-existent directory.

Reported by:	ae
MFC after:	1 week
2017-10-12 13:59:23 +00:00
Enji Cooper
d511b20a69 Add HAS_TESTS to all Makefiles that are currently using the
`SUBDIR.${MK_TESTS}+= tests` idiom.

This is a follow up to r321912.
2017-08-02 08:50:42 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Warner Losh
a35f04fba2 Adopt SRCTOP in usr.bin
Prefer ${SRCTOP}/foo over ${.CURDIR}/../../foo and ${SRCTOP}/usr.bin/foo
over ${.CURDIR}/../foo for paths in Makefiles.

Differential Revision:	https://reviews.freebsd.org/D9932
Sponsored by:		Netflix
Silence on:		arch@ (twice)
2017-03-12 18:58:44 +00:00
Enji Cooper
3947900cdc Restore some of the error message text accidentally removed in r315098
"unexpected symlink contents" is more pedantically correct than
"unexpected symlink".

MFC after:	1 week
X-MFC with:	r315098
Sponsored by:	Dell EMC Isilon
2017-03-12 04:08:36 +00:00
Enji Cooper
ec0ada9dc4 Add 3 more testcases demonstrating how install -l sr works
The additional testcases use absolute paths for sources and targets,
as the other testcase which tested `-l sr` used flat relative paths in
the same directory.

Please note that these testcases do not test `-l a` -- that's already
addressed in the battery of tests.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-12 03:58:54 +00:00
Enji Cooper
ef74be51be Clarify src vs dest path mismatch in :symbolic_link_{absolute,relative}_body
Unfortunately kyua does not omit the path mismatch on failure, so it must be coded
into the error message.

Cache the values, run the test(1) call, then print out the values in an atf_fail
call to emit the required diagnostics to debug why things are failing.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-12 03:29:49 +00:00
Bryan Drewery
1eb4ec9c62 DIRDEPS_BUILD: Add some missing dirctories to the build.
Sponsored by:	EMC / Isilon Storage Division
2016-08-31 19:30:59 +00:00
Bryan Drewery
c8a8570cdb Squelch a false-positive Clang static analyzer warning.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-10 18:19:14 +00:00
Bryan Drewery
4a4768d948 Fix -b failure not restoring flags on the destination file.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-10 18:19:11 +00:00
Bryan Drewery
89ad63ad2d Fix -S with -l not being atomic.
It was unlinking the target even though it uses rename(2) which already
effectively does that.  -S is intended to not unlink(2) the target first.

MFC after:	1 week
Reviewed by:	jhb
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D7452
2016-08-10 18:19:08 +00:00
Bryan Drewery
49cfff9131 Support -v for -l.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2016-08-10 18:19:05 +00:00
Bryan Drewery
b81ddc83c9 Fix -S with -b not atomically updating the destination file.
With both of these flags, the backup was created via rename(dest, backup)
followed by rename(tmp, dest).  This left the destination file missing
for a moment which contradicts the point of -S.

This fixes a race with installworld where PRECIOUSPROG and PRECIOUSLIB
files (which use -S for installation) would briefly be missing.  In the
case of installing rtld with parallel installworld it could render an
error due to not having rtld present to run install/cp in another
process.

Reported by:	jhb
Reviewed by:	jhb
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D7451
2016-08-10 18:19:02 +00:00
Bryan Drewery
e9a4487309 Trim out excessive / with -v when target directory ends with a trailing '/'.
This is a minor nit after r289391 made all installations to a directory always
end in a trailing '/'.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-10 18:18:51 +00:00
Ed Schouten
491cb043bd Pull a copy of the input string before calling basename() and dirname().
POSIX allows implementations of these functions to modify their input.
2016-07-28 16:06:37 +00:00
Jilles Tjoelker
649a3a5626 install: When preserving timestamps, also copy the nanoseconds part.
Now that we have utimensat in -legacy, install(1) can use it.

This is a revert of r299942 which is itself a revert of r299850.
2016-06-09 21:59:35 +00:00
Jilles Tjoelker
dd53735797 install: Revert utimensat usage (r299850).
This should fix the build on older stable/10, since install is a bootstrap
tool.

Pending a decision how to fix this properly, revert utimensat usage. Copies
with the -p option will again appear older than the original almost always,
but -p is not commonly used.
2016-05-16 12:56:28 +00:00
Jilles Tjoelker
ecaa1101c3 install: When preserving timestamps, also copy the nanoseconds part. 2016-05-15 13:17:05 +00:00
Jilles Tjoelker
1f9b8f8e0d install: Add some tests. 2016-05-08 21:11:24 +00:00
Bryan Drewery
5a0d38b3aa META MODE: Remove unneeded libmd.host dependency for xinstall.host.
This is chasing r291026.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:55 +00:00
Bryan Drewery
4aa63711a7 META MODE: Fix 'make bootstrap-tools'.
The main problem was bitrot after elftoolchain being swapped in for the
GNU toolchain.

This also reworks how the list of 'host allowed' libraries is determined
to only allow INTERNALLIBs, which is needed for libelftc to come in.

For usr.bin/readelf use the same hack, as libelf and libdward, to bring in
the needed sys/ headers for host builds.  This has not yet been a problem due
to readelf not being built as a host tool in buildworld.  This is possible
in the meta build though when building the toolchain.

Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:44:51 +00:00
Baptiste Daroussin
d0b9260683 install: do not follow symlinks
In case the target of install is a dead symlink, install(1) used to not
consider it as "existing" because of the usage of stat(2) instead of
lstat(2).  meaning the old file (the symlink) is not removed before the new
file is created. The symlink is being followed and the new file becoming the
target of the symlink instead of the target of install(1)

Reviewed by:	jhb, brooks
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D4191
2015-11-20 08:45:59 +00:00
Bryan Drewery
20f680588a Remove unneeded libmd from bootstrap-tools (reverting r246784).
The bootstrap-tools are supposed to be host tools, which in most cases, use
host headers and libraries.  As such, directly including the src tree's headers
for libmd here causes the need to link libmd in since it will be built with
the new symbols (which /usr/lib/libmd.so) won't have unless it is new enough.
During the target build in buildworld the target headers are staged into
WORLDTMP and used via --sysroot, allowing the target xinstall to be built with
the new/target libmd.

The .PATH here was also not doing anything since xinstall does not use libmd
source files.

Sponsored by:	EMC / Isilon Storage Division
MFC after:	2 weeks
2015-11-18 21:37:46 +00:00
Ed Maste
209e744f89 xinstall: renumber UCB clauses to avoid skipping #3 2015-09-25 20:25:18 +00:00
Baptiste Daroussin
9ebcef9a74 short circuit install -l rs
When requesting install(1) to only make relative symlinks, by pass all the
done to actually compute the relative symlink if the path given in argument is
already a relative path
2015-06-26 23:55:02 +00:00
Simon J. Gerraty
2ef6d5a7b9 new depends 2015-06-16 23:37:19 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Baptiste Daroussin
3e11bd9e2a Convert to usr.bin/ to LIBADD
Reduce overlinking
2014-11-25 14:29:10 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Mateusz Guzik
6e633f7297 install: re-check failed mkdir for EEXIST
Since the code stats and mkdirs in 2 separate steps, it is possible that
the directory will be created in the meantime by something else (e.g.
concurrent install).[1]

While here alter the code to properly report stat failure, previously it
would always claim it was mkdir which failed.

Noted by: royger [1]
MFC after:	1 week
2014-09-23 11:41:09 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
3b8f084595 Merge head 2014-04-28 07:50:45 +00:00
Jilles Tjoelker
236a5a7938 install: Use posix_spawnp() for starting strip and improve error messages. 2014-03-08 23:05:28 +00:00