Commit Graph

248866 Commits

Author SHA1 Message Date
kevans
7329b74072 stand/zfs: Add all bootenvs to environment
For the benefit of lualoader, add all bootenvs to environment when
init_zfs_bootenv is invoked. All of the boot environment logic can then be
implemented in pure lua, rather than going back and forth with C to
implement paging.

This stores all boot environments in bootenvs[idx] and the final count of
bootenvs in bootenvs_count.

While here, make a copy of currdev for init_zfs_bootenv since it will be
modifying it and the caller may not necessarily want that. Some of the logic
was shifted around so that the 'currdev' pointer remains at the beginning of
the string and 'beroot' is moved around as needed to modify it or ultimately
store it in zfs_be_root.

The original zfs_bootenv that this was copied from will be able to go away
only if/when forth eventually goes away.

Tested with:	lualoader (and local changes to add boot env. support)
Tested with:	forth
Reviewed by:	cem (earlier version), imp
Differential Revision:	https://reviews.freebsd.org/D14435
2018-02-21 16:26:16 +00:00
imp
ad4101f603 Honor settings for including / excluding cd9660, ufs, ext2fs and msdos.
The Makefile gives the impression that ext2fs and msdos were excluded
(they weren't) and that you could exclude cd9660 and ufs support (you
couldn't). Allow those to be excluded.

We need to look, in the future, at trimming the number of supported
filesystems, and this will make that easier.
2018-02-21 15:58:00 +00:00
imp
df01c2b09b Consolidate three copies of ZFS commands into a central location.
There's no reason to have multiple copies of lszfs and
reloadbe. Consolidate them into one location. Also ldi_get_size is the
same everywhere (except sparc64). Make it the same everywhere as the
common definition is more general and will work on spar64.
2018-02-21 15:57:36 +00:00
trasz
784e0cc049 Build getty(8) with WARNS=6.
Reviewed by:	imp@
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14197
2018-02-21 15:57:24 +00:00
emaste
b3fd6ddf73 ministat: disallow negative variance / nan Stddev
With all values identical it was possible for Var() to return a negative
value due to limited floating point precision, resulting in "nan"
reported as Stddev.

Variance cannot actually be negative, so just return 0.  We can later
investigate alternate algorithms for calculating variance to reduce the
effect of catastrophic cancellation here.

Reported by:	Arshan Khanifar <arshankhanifar_gmail.com>
Approved by:	phk
Sponsored by:	The FreeBSD Foundation
2018-02-21 15:54:23 +00:00
asomers
b53f4cb1fe zdb: raise WARNS from 0 to 2
This has only been possible since r329694 and r329508

MFC after:	3 weeks
X-MFC-With:	329694
X-MFC-With:	329508
Sponsored by:	Spectra Logic
2018-02-21 15:51:48 +00:00
nwhitehorn
4f19c9bbb9 Add definition for the PowerPC A2. 2018-02-21 15:15:58 +00:00
nwhitehorn
51f7eaccc2 Add definitions for the new Radix MMU mode on POWER9+ CPUs. 2018-02-21 15:15:31 +00:00
avg
d6fce51483 MFV r329718: 8520 7198 lzc_rollback_to should support rolling back to origin
illumos/illumos-gate@95643f75d2
95643f75d2

https://www.illumos.org/issues/8520
  lzc_rollback_to() should support rolling back to a clone's origin.
  The current checks in zfs_ioc_rollback() would not allow that because the
  origin snapshot belongs to a different filesystem.
  The overly restrictive check was introduced in 7600, but it was not a
  regression as none of the existing tools provided a way to rollback to the
  origin.

https://www.illumos.org/issues/7198
  EINVAL is returned when a dataset does not have any snapshots, so there is
  nothing to roll back to.
  Although the code in zfs_do_rollback checks for that condition in advance, it's
  still possible that the snapshot(s) gets removed after the check and before the
  rollback sync task is executed.
  At the moment zfs command would crash when that happens.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	2 weeks
2018-02-21 15:12:14 +00:00
avg
4ac98f46ac MFV r329715: 8997 ztest assertion failure in zil_lwb_write_issue
illumos/illumos-gate@f864f99efe
f864f99efe

https://www.illumos.org/issues/8997
  When dmu_tx_assign is called from zil_lwb_write_issue, it's possible
  for either ERESTART or EIO to be returned.
  If ERESTART is returned, this will cause an assertion to fail directly
  in zil_lwb_write_issue, where the code assumes the return value is
  EIO if dmu_tx_assign returns a non-zero value. This can occur if the
  SPA is suspended when dmu_tx_assign is called, and most often occurs
  when running zloop.
  If EIO is returned, this can cause assertions to fail elsewhere in the
  ZIL code. For example, zil_commit_waiter_timeout contains the
  following logic:
    lwb_t *nlwb = zil_lwb_write_issue(zilog, lwb);
    ASSERT3S(lwb->lwb_state, !=, LWB_STATE_OPENED);
  In this case, if dmu_tx_assign returned EIO from within
  zil_lwb_write_issue, the lwb variable passed in will not be issued
  to disk. Thus, it's lwb_state field will remain LWB_STATE_OPENED and
  this assertion will fail. zil_commit_waiter_timeout assumes that after
  it calls zil_lwb_write_issue, the lwb will be issued to disk, and
  doesn't handle the case where this is not true; i.e. it doesn't handle
  the case where dmu_tx_assign returns EIO.

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Prakash Surya <prakash.surya@delphix.com>
MFC after:	3 weeks
2018-02-21 15:07:49 +00:00
kevans
36c22f5a5e lualoader: Use the key that interrupts autoboot as a menu choice
This matches forth behavior. Hitting "6" when autobooting at the welcome
menu will now take you directly to the "Boot Options" menu.

We likely have some slight optimizations we should make, like not checking
autoboot every time we open a new menu and things of this nature. Further
work will go towards this end.
2018-02-21 14:37:49 +00:00
avg
cbfdd7dd99 MFV r329713: 8731 ASSERT3U(nui64s, <=, UINT16_MAX) fails for large blocks
illumos/illumos-gate@a6c1eb3c08
a6c1eb3c08

https://www.illumos.org/issues/8731
  annotate_ecksum() asserts that nui64s, calculated as nui64s = size / sizeof
  (uint64_t), is not greater than UINT16_MAX.
  This restriction is needed because histograms of incorrectly set and cleared
  bits have 16 bit counters and if the buffer consists of too many 64-bit words,
  then a counter can potentially overflow producing an incorrect result.
  When the largest buffer size was 128KB the greatest value of nui64s was 16K,
  well within the limit.
  But now we have support for large buffers and for buffer sizes of 512KB and
  above the restriction is violated.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	2 weeks
2018-02-21 14:31:48 +00:00
wma
7908daef61 PowerNV: Put processor to power-save state in idle thread
When processor enters power-save state it releases resources shared with other
cpu threads which makes other cores working much faster.

This patch also implements saving and restoring registers that might get
corrupted in power-save state.

Submitted by:          Patryk Duda <pdk@semihalf.com>
Obtained from:         Semihalf
Reviewed by:           jhibbits, nwhitehorn, wma
Sponsored by:          IBM, QCM Technologies
Differential revision: https://reviews.freebsd.org/D14330
2018-02-21 14:28:40 +00:00
avg
9bdcb4a200 MFV r329710: 8966 Source file zfs_acl.c, function zfs_aclset_common contains a use after end of the lifetime of a local variable
illumos/illumos-gate@82693e09cc
82693e09cc
https://www.illumos.org/issues/8966

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: WHR <msl0000023508@gmail.com>
PR:		225162
Submitted by:	WHR <msl0000023508@gmail.com>
Reported by:	WHR <msl0000023508@gmail.com>
MFC after:	1 week
2018-02-21 14:17:07 +00:00
kevans
b98e65b863 lualoader: Don't autodetect kernels if 'kernels' is explicitly set 2018-02-21 14:07:53 +00:00
trasz
6ccea74d8c Use proper buffer length (the announce_buf char pointer used to be anarray),
broken in r317143. This fixes those weird "cd0: Attempt" messages at boot.

PR:		222103
Reviewed by:	scottl@
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14369
2018-02-21 14:05:13 +00:00
hselasky
ef80f6306d Allow LinuxKPI character devices to receive mmap() calls from the Linux
binary mode user-space emulation layer. This is a regression issue after
r328436, when LinuxKPI character devices started to use DTYPE_DEV in
the "f_type" field of the associated file structure(s).

MFC after:	3 days
Found by:	Johannes Lundberg <johalun0@gmail.com>
Sponsored by:	Mellanox Technologies
2018-02-21 10:13:17 +00:00
wma
916a28b415 PowerNV: add missing RTC_WRITE support
Add function which can store RTC values to OPAL.

Submitted by:          Wojciech Macek <wma@semihalf.org>
Obtained from:         Semihalf
Sponsored by:          IBM, QCM Technologies
2018-02-21 08:13:17 +00:00
wma
427b6274ff CXGBE: implement prefetch on non-Intel architectures
Submitted by:          Michal Stanek <mst@semihalf.com>
Obtained from:         Semihalf
Reviewed by:           np, pdk@semihalf.com
Sponsored by:          IBM, QCM Technologies
Differential revision: https://reviews.freebsd.org/D14452
2018-02-21 08:05:56 +00:00
kevans
fc3ba7b848 lualoader: Allow carousel 'items' to be a table as well as a function
We don't have any in-tree users of this, but for a static set of carousel
options having to define a callback is excessive.
2018-02-21 05:04:58 +00:00
kevans
3462059e6f lualoader: Simplify menu definitions a little further
Allow "name" entries to be simple strings, instead of just functions. We
know whether we support colors or not by the time any of this is setup, so
all menu names that are basically static with colors sprinkled in are good
candidates for simplification.

Also simplify "func" in many cases where it's just invoking another function
with no arguments. The downside to this simplification is that the functions
called can no longer be trivially replaced by a local module. The upside is
that it removes another layer of indirection that we likely don't need.

These can be re-evaluated later if a compelling argument is raised, on a
case-by-case basis, for replacement.
2018-02-21 04:48:37 +00:00
kevans
04ed775ac8 lualoader: Directly reference submenu definition with submenu key 2018-02-21 04:23:13 +00:00
kevans
560b96d480 lualoader: Drop name requirement for menu separators 2018-02-21 04:14:32 +00:00
kevans
fd1c4e5bd7 lualoader: Add "menu.default", initialized to menu.welcome
The intent here is to abstract away the name of the default menu. The
default menu is still the welcome menu, but this detail doesn't need to
matter to things outside of the menu module. You may change the default
menu, but one would need to modify a specific menu.
2018-02-21 03:37:46 +00:00
jhibbits
225ff95e18 Split printtrap() into generic and CPU-specific components
Summary:
This compartmentalizes the CPU-specific trap components into its own
function, rather than littering the general printtrap() with various checks.
This will let us replace a series of #ifdef's with a runtime conditional check
in the future.

Reviewed By:	nwhitehorn
Differential Revision:	https://reviews.freebsd.org/D14416
2018-02-21 03:34:33 +00:00
mav
7704db0f8f MFV r324198: 8081 Compiler warnings in zdb
illumos/illumos-gate@3f7978d02b
3f7978d02b

https://www.illumos.org/issues/8081
  zdb(8) is full of minor problems that generate compiler warnings. On FreeBSD,
  which uses -WError, the only way to build it is to disable all compiler
  warnings. This makes it much harder to detect newly introduced bugs. We should
  cleanup all the warnings.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Alan Somers <asomers@gmail.com>
2018-02-21 03:08:47 +00:00
kevans
0923320a01 lualoader: Return only argstr if with_kernel not requested 2018-02-21 03:06:36 +00:00
kevans
abc98ada75 lualoader: Bring in local.lua module if it exists
Provide a way for out-of-tree users of lualoader to patch into the loader
system without having to modify our distributed scripts.

Do note that we can't really offer any API compatibility guarantees at this
time due to the evolving nature of lualoader right now.

This still has some utility as local modules may add commands at the loader
prompt without relying heavily on lualoader features- this specific
functionality is less likely to change without more careful consideration.

Reviewed by:	cem (earlier version)
Differential Revision:	https://reviews.freebsd.org/D14439
2018-02-21 02:35:13 +00:00
mav
484c4a76cc MFV r322231:
8430 dir_is_empty_readdir() doesn't properly handle error from fdopendir()

illumos/illumos-gate@ba6e7e6505
ba6e7e6505

https://www.illumos.org/issues/8430
  we should close dirfd if fdopendir() fails.

Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Sowrabha Gopal <sowrabha.gopal@delphix.com>
2018-02-21 02:21:22 +00:00
mav
d433208386 MFV r319737: 6939 add sysevents to zfs core for commands
illumos/illumos-gate@ce1577b049
ce1577b049

https://www.illumos.org/issues/6939
  Originally created https://smartos.org/bugview/OS-4489
       sysevents should be fired in the kernel from ZFS whenever a command
       is run that is logged in zpool history.
  Example output
  Terminal 1
  root - gz sunos ~ # zfs create zones/foobar
  root - gz sunos ~ # zfs set quota=10g zones/foobar
  root - gz sunos ~ # zfs destroy zones/foobar
  Terminal 2
  root - gz sunos ~ # sysevent EC_zfs
  nvlist version: 0
      date = 2016-04-28T14:50:08.964Z
      vendor = SUNW
      publisher = zfs
      class = EC_zfs
      subclass = ESC_ZFS_history_event
      pid = 0
      data = (embedded nvlist)
      nvlist version: 0
          pool_name = zones
          pool_guid = 0x40c964e8f9a7a694
          history_record = (embedded nvlist)
          nvlist version: 0
              dsname = zones/foobar
              dsid = 0x1525
              history internal str =
              internal_name = create
              history txg = 0x4c4ef3

Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Joshua M. Clulow <jmc@joyent.com>
Reviewed by: Josh Wilsdon <jwilsdon@joyent.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Reviewed by: Alan Somers <asomers@gmail.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Dave Eddy <dave@daveeddy.com>
2018-02-21 02:19:42 +00:00
kevans
9f61801c50 lualoader: Output "Failed to parse" messages
I can't find any good reason these aren't enabled, so enable them.

The silent runs will only return false on actual parse errors, so it's ok to
be loud about those failures.
2018-02-21 01:52:42 +00:00
kevans
5113a84d0c lualoader: Don't return false for failure to open config on silent parse 2018-02-21 01:50:46 +00:00
kevans
6cc4d46d2b lualoader: Drop explicit boolean tests; b or not b 2018-02-21 01:39:33 +00:00
kevans
6ce39f6cb7 style.lua(9): Drop notes about semicolons
It is generally agreed upon that they are ugly and should not be used except
where needed.
2018-02-21 01:37:22 +00:00
kevans
ea0bbcce59 lualoader: Drop excessive parenthesizing
This was also a convenience convention (for me) that is not very lua-tic.
Drop it.

I've maintained some parentheses where I'd prefer them, for example,
'if x or y or (z and w) then', but these situations are far and few between.
2018-02-21 01:35:19 +00:00
kevans
8b04ad2da2 lualoader: Drop terminating semicolons
This was previously chosen out of convenience, as we had a mixed style and
needed to be consistent. I started learning Lua on Friday, so I switched
everything over. It is not a very lua-nic convention, though, so drop it.

Excessive parenthesizing around conditionals is next on the chopping block.
2018-02-21 01:10:03 +00:00
mav
65ce973bc5 MFV r319736: 6396 remove SVM
illumos/illumos-gate@5f10ef697f
5f10ef697f

https://www.illumos.org/issues/6396
  LVM = SVM = Solaris Volume Manager
  dead code and not using with ZFS based platform.

Reviewed by: Igor Kozhukhov <ikozhukhov@gmail.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
Author: Yuri Pankov <yuri.pankov@nexenta.com>
2018-02-21 00:24:54 +00:00
rpokala
739005fe70 mountd: Return proper errno values in a few error paths
When attempting to mount a non-directory which exists, return ENOTDIR instead
of ENOENT. If stat() or statfs() failed, don't pass part of the invalid
(struct statfs) to ex_search(). In that same case, preserve the value of "bad"
rather than overwriting with EACCES.

Submitted by:	Bruce Leverett (Panasas)
Reviewed by:	rmacklem
MFC after:	1 week
Sponsored by:	Panasas
Differential Revision:	https://reviews.freebsd.org/D14438
2018-02-21 00:19:02 +00:00
mav
eb978148c0 MFV r318941: 7446 zpool create should support efi system partition
illumos/illumos-gate@7855d95b30
7855d95b30

https://www.illumos.org/issues/7446
  Since we support whole-disk configuration for boot pool, we also will need
  whole disk support with UEFI boot and for this, zpool create should create efi-
  system partition.
  I have borrowed the idea from oracle solaris, and introducing zpool create -
  B switch to provide an way to specify that boot partition should be created.
  However, there is still an question, how big should the system partition be.
  For time being, I have set default size 256MB (thats minimum size for FAT32
  with 4k blocks). To support custom size, the set on creation "bootsize"
  property is created and so the custom size can be set as: zpool create B -
  o bootsize=34MB rpool c0t0d0
  After pool is created, the "bootsize" property is read only. When -B switch is
  not used, the bootsize defaults to 0 and is shown in zpool get output with
  value ''. Older zfs/zpool implementations are ignoring this property.
  https://www.illumos.org/rb/r/219/

Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Toomas Soome <tsoome@me.com>

This commit makes no sense for FreeBSD, that is why I blocked the option,
but it should be good to stay closer to upstream.
2018-02-21 00:18:57 +00:00
kevans
eb26919d33 lualoader: When restoring environment, only restore unchanged vars
Track the latest value we've set an environment variable to, and only
restore those that are unchanged from that.

This gives us some leeway to make sure we're not clobbering variables
overwritten by menu changes.
2018-02-20 22:47:49 +00:00
bdrewery
0f6e4bdd0f Properly lookup values if they were empty.
Spotted by:	rpokala
X-MFC-With:	r329676
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:08 +00:00
bdrewery
c820d3befe Only lookup SVN once and rename to SVN_CMD.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:04 +00:00
bdrewery
554b0c9dcb Avoid looking up VCS_REVISION for every sub-make call to Makefile.inc1.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 21:53:39 +00:00
bdrewery
5d18971cc6 Move SVNVERSION_CMD into the one place that uses it.
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk.  This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.

Sponsored by:	Dell EMC
MFC after:	2 weeks
2018-02-20 21:48:16 +00:00
np
a38188ac36 Catch up with the removal of nktr_slot_flags from upstream netmap. No
functional impact intended.

Submitted by:	Vincenzo Maffione <v.maffione@gmail.com>
2018-02-20 21:42:45 +00:00
kevans
c7507a4c53 lualoader: Intercept the 'autoboot' cli command 2018-02-20 21:37:55 +00:00
kevans
fca6cee8a1 lualoader: Intercept boot cli command
This should be functional and roughly equivalent to the Forth version.

Stop doing a loadelf() on menu exit now that we can DTRT with boot
invocations. autoboot interception will follow not long after.
2018-02-20 21:32:36 +00:00
jeff
21bd298f13 Fix the broken subqueue assignment for the cleanq.
Reported by:	pho
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
2018-02-20 21:27:17 +00:00
kevans
2cd022e913 lualoader: Prepare for interception of "boot" CLI cmd
core.boot and core.autoboot may both take arguments; add a helper to cleanly
append an argstring to the given loader command.

Also provide a popFrontTable() that we'll use pop the command name off of an
argv table. We don't have the table library included, and including it is
non-trivial, so we'll implement this one function that we need in lua for
the time being.
2018-02-20 21:23:01 +00:00
kevans
24e1c0efcc lualoader: Don't try so hard to load a kernel
If the user's selected a kernel, we really should be trying to load that one
instead of falling back to some default kernel.

This should generally be a no-op and most desirable, unless you really
enjoyed surprises.
2018-02-20 21:15:43 +00:00