freebsd-nq/tests/zfs-tests/include
Matthew Ahrens 3eabed74c0
Fix lua stack overflow on recursive call to gsub()
The `zfs program` subcommand invokes a LUA interpreter to run ZFS
"channel programs".  This interpreter runs in a constrained environment,
with defined memory limits.  The LUA stack (used for LUA functions that
call each other) is allocated in the kernel's heap, and is limited by
the `-m MEMORY-LIMIT` flag and the `zfs_lua_max_memlimit` module
parameter.  The C stack is used by certain LUA features that are
implemented in C.  The C stack is limited by `LUAI_MAXCCALLS=20`, which
limits call depth.

Some LUA C calls use more stack space than others, and `gsub()` uses an
unusually large amount.  With a programming trick, it can be invoked
recursively using the C stack (rather than the LUA stack).  This
overflows the 16KB Linux kernel stack after about 11 iterations, less
than the limit of 20.

One solution would be to decrease `LUAI_MAXCCALLS`.  This could be made
to work, but it has a few drawbacks:

1. The existing test suite does not pass with `LUAI_MAXCCALLS=10`.

2. There may be other LUA functions that use a lot of stack space, and
the stack space may change depending on compiler version and options.

This commit addresses the problem by adding a new limit on the amount of
free space (in bytes) remaining on the C stack while running the LUA
interpreter: `LUAI_MINCSTACK=4096`.  If there is less than this amount
of stack space remaining, a LUA runtime error is generated.

Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Ryan Moeller <ryan@ixsystems.com>
Reviewed-by: Allan Jude <allanjude@freebsd.org>
Reviewed-by: Serapheim Dimitropoulos <serapheim@delphix.com>
Signed-off-by: Matthew Ahrens <mahrens@delphix.com>
Closes #10611 
Closes #10613
2020-07-27 16:11:47 -07:00
..
.gitignore Add zpool events tests 2017-05-22 12:34:42 -04:00
blkdev.shlib ZTS: Fix nonportable use of stat in list_file_blocks 2020-07-15 21:26:39 -07:00
commands.cfg Centralize variable substitution 2020-07-14 17:33:44 -07:00
default.cfg.in ZTS: Fix a few defaults 2020-01-31 08:51:23 -08:00
libtest.shlib Remove dependency on sharetab file and refactor sharing logic 2020-07-13 09:19:18 -07:00
Makefile.am Centralize variable substitution 2020-07-14 17:33:44 -07:00
math.shlib ZTS: Make bc conditional use compatible with new BSD bc 2020-07-09 17:49:02 -07:00
properties.shlib ZTS: Don't use edonr on FreeBSD 2020-01-28 08:38:02 -08:00
tunables.cfg Fix lua stack overflow on recursive call to gsub() 2020-07-27 16:11:47 -07:00
zpool_script.shlib zfs-tests: fix warnings when packaging some .shlib files 2019-05-24 14:12:14 -07:00