Commit Graph

17676 Commits

Author SHA1 Message Date
Kyle Evans
417d105fae Revert r336011,r336012 until I can competently test 2018-07-05 18:55:42 +00:00
Kyle Evans
f1e0a986a8 Fix build after r336011
Add libnv to bootstrap-tools, use ${SRCTOP}/sys headers.
2018-07-05 18:39:02 +00:00
Kyle Evans
51c854f028 config(8): De-dupe hint/env vars within a single file
r335653 flipped the order in which hints/env files are concatenated to match
the order in which vars are processed by the kernel. This is the other
hammer to drop.

Use nv(9) to de-dupe entries within a single `hint` or `env` file, using the
latest value specified for a key. This leaves some duplicates if a variable
is specified in multiple hint/env files or via `envvar` in a kernel config,
but the reversed order of concatenation (from r335653) makes this a
non-issue as the latest-specified version will be seen first.

This change also silently rewrote hint bits to use the same sanitization
process that ian@ wrote for r335642. To the kernel, hints and env vars are
basically the same thing through early boot, then get merged into the
dynamic environment once kmem becomes available and the dynamic environment
is created. They should be subjected to the same restrictions.

MFC after:	1 month
2018-07-05 17:53:51 +00:00
Brooks Davis
f6293d7296 Work around lame warnings in ancient gcc on 32-bit platforms.
Fixes r335979.
2018-07-05 17:02:10 +00:00
Kyle Evans
39d44f7f15 kern_environment: use any provided environments, evict hintmode/envmode
At the moment, hintmode and envmode are used to indicate whether static
hints or static env have been provided in the kernel config(5) and the
static versions are mutually exclusive with loader(8)-provided environment.
hintmode *can* be reconfigured later to pull from the dynamic environment,
thus taking advantage of the loader(8) or post-kmem environment setting.

This changeset fixes both problems at once to move us from a semi-confusing
state to a consistent state: if an environment file, hints file, or
loader(8) environment are provided, we use them in a well-known order of
precedence:

- loader(8) environment
- static environment
- static hints file

Once the dynamic environment is setup this becomes a moot point. The
loader(8) and static environments are merged (respecting the above order of
precedence), and the static hints are merged in on an as-needed basis after
the dynamic environment has been setup.

Hints lookup are changed to respect all of the above. Before the dynamic
environment is setup, lookups use the above-mentioned order and fallback to
the next environment if a matching hint is not found. Once the dynamic
environment is setup, that is used on its own since it captures all of the
above information plus any dynamic kenv settings that came up later in boot.

The following tangentially related changes were made to res_find:

- A hintp cookie is now passed in so that related searches continue using
  the chain of environments (or dynamic environment) without relying on
  global state
- All three environments will be searched if they actually have valid hints
  to use, rather than just choosing the first environment that actually had
  a hint and rolling with that only

The hintmode sysctl has been ripped out. static_{env,hints}.disabled are
still honored and will disable their respective environments from being used
for hint lookups and from being merged into the dynamic environment, as
expected.

MFC after:	1 month (maybe)
Differential Revision:	https://reviews.freebsd.org/D15953
2018-07-05 16:30:32 +00:00
Kyle Evans
e28687347f Revert r335995 due to accidental changes snuck in 2018-07-05 16:28:43 +00:00
Kyle Evans
8ef5886303 kern_environment: use any provided environments, evict hintmode/envmode
At the moment, hintmode and envmode are used to indicate whether static
hints or static env have been provided in the kernel config(5) and the
static versions are mutually exclusive with loader(8)-provided environment.
hintmode *can* be reconfigured later to pull from the dynamic environment,
thus taking advantage of the loader(8) or post-kmem environment setting.

This changeset fixes both problems at once to move us from a semi-confusing
state to a consistent state: if an environment file, hints file, or
loader(8) environment are provided, we use them in a well-known order of
precedence:

- loader(8) environment
- static environment
- static hints file

Once the dynamic environment is setup this becomes a moot point. The
loader(8) and static environments are merged (respecting the above order of
precedence), and the static hints are merged in on an as-needed basis after
the dynamic environment has been setup.

Hints lookup are changed to respect all of the above. Before the dynamic
environment is setup, lookups use the above-mentioned order and fallback to
the next environment if a matching hint is not found. Once the dynamic
environment is setup, that is used on its own since it captures all of the
above information plus any dynamic kenv settings that came up later in boot.

The following tangentially related changes were made to res_find:

- A hintp cookie is now passed in so that related searches continue using
  the chain of environments (or dynamic environment) without relying on
  global state
- All three environments will be searched if they actually have valid hints
  to use, rather than just choosing the first environment that actually had
  a hint and rolling with that only

The hintmode sysctl has been ripped out. static_{env,hints}.disabled are
still honored and will disable their respective environments from being used
for hint lookups and from being merged into the dynamic environment, as
expected.

MFC after:	1 month (maybe)
Differential Revision:	https://reviews.freebsd.org/D15953
2018-07-05 16:25:48 +00:00
Marcelo Araujo
c066c68c57 - Add bhyve NVMe device emulation.
The initial work on bhyve NVMe device emulation was done by the GSoC student
Shunsuke Mie and was heavily modified in performan, functionality and
guest support by Leon Dang.

bhyve:
	-s <n>,nvme,devpath,maxq=#,qsz=#,ioslots=#,sectsz=#,ser=A-Z

	accepted devpath:
		/dev/blockdev
		/path/to/image
		ram=size_in_MiB

Tested with guest OS: FreeBSD Head, Linux Fedora fc27, Ubuntu 18.04,
                      OpenSuse 15.0, Windows Server 2016 Datacenter.
Tested with all accepted device paths: Real nvme, zdev and also with ram.
Tested on: AMD Ryzen Threadripper 1950X 16-Core Processor and
           Intel(R) Xeon(R) CPU E5-2609 v2 @ 2.50GHz.

Tests at: https://people.freebsd.org/~araujo/bhyve_nvme/nvme.txt

Submitted by:	Shunsuke Mie <sux2mfgj_gmail.com>,
		Leon Dang <leon_digitalmsx.com>
Reviewed by:	chuck (early version), grehan
Relnotes:	Yes
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D14022
2018-07-05 03:33:58 +00:00
Hartmut Brandt
8e9b3e7071 Update bsnmp to version 1.13. This does not bring user-visible changes.
For developers gensnmptree can now generate functions for enums to convert
between enums and strings and to check the validity of a value.
The sources in FreeBSD are now in sync with the upstream which allows to
bring in IPv6 modifications.
2018-07-03 08:44:40 +00:00
Rick Macklem
0479387e0a Document the "#mds_path" suffix for the "-p" command line option.
r335871 added support for an optional suffix of "#mds_path" that can be
applied to each entry in the "-p" option argument. This specifies that
the DS should be used to store files for the file system on the MDS
at "mds_path".
This patch documents this optional suffix.

This is a content change.
2018-07-02 19:33:26 +00:00
Rick Macklem
ed46427268 Add an optional feature to the pNFS server.
Without this patch, the pNFS server distributes the data storage files across
all of the specified DSs.
A tester noted that it would be nice if a system administrator could control
which DSs are used to store the file data for a given exported MDS file system.
This patch adds an optional suffix for each entry in the "-p" option argument
that specifies "store file data for this MDS file system" in this DS.
The patch should only affect sites using the pNFS server (specified via the
"-p" command line option for nfsd.
The interface between the nfsd and the kernel has changed with this patch,
so anyone using the "-p" option needs to rebuild their nfsd from sources
with this patch applied to them.

Discussed with:	james.rose@framestore.com
2018-07-02 19:26:31 +00:00
Ed Schouten
8ab684e234 Document that syslogd -v has no effect when RFC 5424 mode is enabled.
The variable it sets, LogFacPri, is only used in the RFC 3164 formatting
codepath.

PR:		229457
Reported by:	Andre Albsmeier
MFC after:	1 week
2018-07-02 11:02:27 +00:00
Ed Schouten
8a66bdb414 Restore the order in which RFC 3164 messages with fac/pri are formatted.
The refactoring of the syslogd code to format messages using iovecs
slightly altered the output of syslogd by placing the facility/priority
after the hostname, as opposed to printing it right before. This change
reverts the behaviour to be consistent with how it was before.

PR:		229457
Reported by:	Andre Albsmeier
MFC after:	1 week
2018-07-02 10:57:22 +00:00
Rick Macklem
b3ccfa8a60 Document the new "-m" command line option for pnfsdsfile(8).
This is a content change.
2018-07-01 17:51:52 +00:00
Rick Macklem
7c126cfa95 Add a new "-m" option to pnfsdsfile(8) to prepare a file for mirroring.
When pnfsdscopymr(8) is used to create a mirror of a file on a mirrored
pNFS service, it expects to find an entry in the extended attribute for
IP address 0.0.0.0.
This patch adds a "-m" option which can be used to create these entrie(s).
It also tightens up the checks for use of incompatible command line options.
2018-07-01 17:49:02 +00:00
Edward Tomasz Napierala
775c9076c7 Add basic examples to portsnap(8).
MFC after:	2 weeks
2018-07-01 16:18:38 +00:00
Rick Macklem
2f800e5593 Add a safety belt assignment to the nfsd for when "-p" isn't specified.
The kernel code assumes that nfsdargs.addr == NULL and nfsdargs.addrlen == 0
when there is no "-p" argument used for starting the nfsd.
This small patch ensures this is the case. In practice, I believe this always
happened, since "nfsdargs" was the last element on the stack for "main()",
but this little patch ensures it will be the case.
Spotted by inspection while adding a new optional field for "-p".
2018-06-30 12:14:20 +00:00
Alex Richardson
53ed3b32ae Fix missing files in METALOG with -DNO_ROOT
By using INSTALL_LINK instead of calling ln during install the files
end up in the METALOG file as well if we use -DNO_ROOT and will be
included in a disk image when using makefs with METALOG as the input.
The other file that was not included in METALOG was /var/db/services.db
which is now also included for -DNO_ROOT.

Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D15665
2018-06-29 21:15:17 +00:00
Eitan Adler
48c7315619 powerd: correct ifdef check for ppc
Reviewed by:	jhibbits
2018-06-27 01:28:09 +00:00
Navdeep Parhar
2293e22112 cxgbetool(8): Reject invalid VLAN values.
Submitted by:	Krishnamraju Eraparaju @ Chelsio
MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-06-26 21:56:06 +00:00
Kyle Evans
67fc3f5608 config(8): part of patch disappeared, don't close ifp at the end 2018-06-26 04:06:49 +00:00
Kyle Evans
6ffdd5ef2f config(8): Flip the order of concatenation for hints and env
As previously noted, kernel's processing of these means that the first
appearance of a hint/variable wins. Flipping the order of concatenation
means that later variables override earlier variables, as expected when one
does:

hints x
hints y

Where perhaps x is:

hint.aw_sid.0.disable=1

and y is:

hint.aw_sid.0.disable=0

The expectation would be that a later appearing variable would override an
earlier appearing variable, such as with `device`/`nodevice`, device.hints,
and other similarly structured data files.
2018-06-26 04:02:25 +00:00
Kyle Evans
4edfa9085a config(8): Make 'env' files consistent with other file-accepting options
Previously, only one 'env' file could be specified. Later 'env' directives
would overwrite earlier 'env' directives. This is inconsistent with every
other file-accepting directives which process files in order, including
hints.

A caveat applies to both hints and env that isn't mentioned: they're
concatenated in the order of appearance, so they're not actually applied in
the way one might think by supplying:

hints x
hints y

Hints in x will take precedence over same-name hints in y due to how
the kernel processes them, stopping at the first line that matches the hint
we're searching for. Future work will flip the order of concatenation so
that later files may still properly override earlier files.

In practice, this likely doesn't matter at all due to the nature of the
beast.
2018-06-26 03:56:10 +00:00
Kyle Evans
1aae236569 config(8): Set envmode if we accept an envvar
X-MFC-With:	r335642
2018-06-26 02:05:45 +00:00
Kyle Evans
3b31596dc8 config(8): Add envvar support
envvar allows adding individual environment variables to the kernel's static
environment without the overhead of pulling in a full file. envvar in a
config looks like:

envvar some_var=5

All envvar-provided variables will be added after the env file is processed,
so envvar keys that exist in the previous env will be overwritten by
whatever value is set here in the kernel configuration directly.

As an aside, envvar lines are intentionally tokenized differently from
basically every other line. We used a named state when ENVVAR is encountered
to gobble up the rest of the line, which will later be cleaned and validated
in post-processing by sanitize_envline. This turns out to be the simplest
and cleanest way to allow the flexibility that kenv does while not
compromising on silly hacks.

Reviewed by:	ian (also contributor of sanitize_envline rewrite)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D15962
2018-06-25 17:17:47 +00:00
Ian Lepore
110d4e2df3 Add an example for displaying the manufacturer and size info from a
standard spiflash chip.
2018-06-23 23:08:25 +00:00
Ed Schouten
531b456983 Still parse messages that don't contain an RFC 3164 timestamp.
The changes made in r326573 required that messages always start with an
RFC 3164 timestamp. It looks like certain devices, but also certain
logging libraries (Python 3's "logging" package) simply don't generate
RFC 3164 formatted messages containing a timestamp.

Make timestamps optional again. When the timestamp is missing, also
assume that the message contains no hostname. The first word of the
message likely already belongs to the message payload.

PR:		229236
Reported by:	Michael Grimm & Marek Zarychta
Reviewed by:	glebius (cursory)
MFC after:	1 week
2018-06-22 20:53:39 +00:00
Ian Lepore
917d184626 Eliminate gcc "shadowed declaration" warnings by using idx rather than
index as a variable name.

Reported by:	manu@
2018-06-22 03:40:03 +00:00
Ian Lepore
6666009530 Add spi(8), a utility for communicating with a device on a SPI bus from
userland, conceptually similar to what i2c(8) provides for i2c devices.

Submitted by:	Bob Frazier
Differential Revision:	https://reviews.freebsd.org/D15029
2018-06-22 01:59:19 +00:00
Bryan Drewery
b7a8f5e16f Let -s actually work.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-22 01:10:05 +00:00
Ed Maste
75e6ea22c5 acpidump.8: include NFIT in the man page list of tables
Was missed in r321298.

Reported by:	Ben Widawsky (in review D15931)
MFC after:	1 week
2018-06-20 19:40:54 +00:00
Mariusz Zaborski
7672a0148f Convert cap_enter() < 0 && errno != ENOSYS to caph_enter() < 0.
No functional change intended.
2018-06-19 23:43:14 +00:00
Sean Bruno
01cebb69be MK_EFI - Add uefisign and friends to this knob and ensure that we don't
try to build them if MK_OPENSSL is unset.

Reviewed by:	emaste imp kevans
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15211
2018-06-19 21:07:25 +00:00
Ed Schouten
8803b8597a Fix bad logic in iovlist_truncate().
To conform to RFC 5426, this function is intended to truncate messages
if they exceed the message size limits. Unfortunately, the amount of
space was computed the wrong way around, causing messages to be
truncated entirely.

Reported by:	Michael Grimm on stable@
MFC after:	3 days
2018-06-18 06:01:28 +00:00
Devin Teske
389e494006 bsdconfig: Fix a bug when editing users
The usermgmt API was stomping on a global ($user_gid to be specific)
so things would appear to work fine until you tried to make a second
pass into the API with the now-tainted variable contents.

Fixed by localizing menu-specific contents as to not leak outside API.

PR:		bin/208774
Reported by:	Martin Waschbuesch <martin@waschbuesch.de>
MFC after:	1 week
X-MFC-to:	stable/11, stable/10
Sponsored by:	Smule, Inc.
2018-06-17 22:09:43 +00:00
Devin Teske
da89782f7d bsdconfig: Make examples optional
PR:		bin/228485
Submitted by:	Dmitry Wagin <dmitry.wagin@ya.ru>
Reported by:	Dmitry Wagin <dmitry.wagin@ya.ru>
MFC after:	1 week
Sponsored by:	Smule, Inc.
2018-06-17 21:09:21 +00:00
Devin Teske
eca1e88c0b sysrc.subr: Fix a comment for accuracy
PR:		bin/203435
Reported by:	Andreas Sommer <andreas.sommer87@googlemail.com>
MFC after:	6 days
X-MFC-to:	stable/11
X-MFC-with:	r335280
Sponsored by:	Smule, Inc.
2018-06-17 20:32:43 +00:00
Alan Somers
f1ed5c000c praudit(1): add tests
Submitted by:	aniketp
MFC after:	2 weeks
X-MFC-With:	335287
Sponsored by:	Google, Inc. (GSoC 2018)
Differential Revision:	https://reviews.freebsd.org/D15751
2018-06-17 17:31:16 +00:00
Devin Teske
5164504a9e sysrc.subr: Fix display when value is "-n"
PR:		bin/226406
Reported by:	Marius Halden <marius.halden@modirum.com>
MFC after:	1 week
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-06-17 06:25:32 +00:00
Devin Teske
181560bbbf sysrc.subr: Fix handling of files with missing newline at EOF
PR:		bin/203435
Reported by:	Andreas Sommer <andreas.sommer87@googlemail.com>
MFC after:	1 week
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-06-17 06:03:48 +00:00
Devin Teske
5c165dcf50 sysrc(8): Exit with failure on API error
Fix exit status when f_sysrc_set() fails. Errors in the underlying API
provided by bsdconfig(8) -- /usr/share/bsdconfig/sysrc.subr -- were not
being communicated back to the command-line. This was affecting ansible
modules using sysrc as they were not able to accurately test for error.

PR:		bin/211448
Reported by:	Christian Schwarz <me@cschwarz.com>
MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-06-17 03:33:29 +00:00
Rick Macklem
a463bac8dd Use the .Fx macro in the man page.
Reported by:	bjk
2018-06-15 21:07:14 +00:00
Rick Macklem
509a73235b Use the .Fx macro in the man page.
Reported by:	bjk
2018-06-15 21:04:28 +00:00
Rick Macklem
4eda4b667a Use the .Fx macro in the man page.
Reported by:	bjk
2018-06-15 21:02:33 +00:00
Rick Macklem
a520a7cf2f Add a command that copies or migrates a data file from one DS to another.
This command can be used by a sysadmin to either copy or migrate a data
file on one DS to another DS.
Its main use is to recover data files onto a mirrored DS after the DS has
been repaired and brought back online.
2018-06-15 19:45:15 +00:00
Rick Macklem
6cb9ec3cc4 Add a command the displays and modifies the pNFS server's extended attribute.
This command allows a sysadmin to display or modify the pnfsd.dsfile extended
attribute used by the pNFS MDS server in various ways.
Its main use is to set a DS's IP address to 0.0.0.0 when that DS has failed,
so that it will not be used for the file when brought back online after
being repaired.
2018-06-15 19:35:08 +00:00
Ed Maste
f756301851 Remove obsolete asf(8)
kgdb now handles kernel module state internally, so the asf tool serves
no purpose.

PR:		229046
Reviewed by:	brooks
Relnotes:	yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15827
2018-06-15 17:44:21 +00:00
Ed Maste
d1ded78d8e Add deprecation notice in asf.8
PR:		229046
Reviewed by:	jhb
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-06-15 17:38:07 +00:00
Rick Macklem
725f388bdc Add an entry into the Makefile for pnfsdskill.
pnfsdskill was added by r335192. This commit adds an entry for it to the
Makefile.
2018-06-15 11:54:55 +00:00
Rick Macklem
b3cd498b58 Add a command that disables a pNFS server mirrored DS.
This command can be used by a sysadmin to disable a malfunctioning pNFS server
mirrored DS. It is safe to use when a mirrored DS has already been disabled
via an I/O or network partitioning error.
2018-06-15 11:52:34 +00:00