The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.
Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.
Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: delphij (previous version), emaste (previous version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4713
wrapped in an i386 ifdef with a comment questioning their usefulness even
there. It turns out they aren't referenced anywhere, but their presence
prevents using sys/endian.h in libstand code.
These days, sys/endian.h provides much better support for such things, using
compiler builtins and inline functions (and creating connections between
libstand code and header files from sys/ would not be breaking new ground).
only happen on every Nth call. Update the existing twiddle() calls done in
various IO loops to roughly reflect the relative IO sizes. That is, tftp
and nfs call twiddle() on every 1K block, ufs on every filesystem block,
so the network calls now use a much larger divisor than disk IO calls.
Also add a new twiddle_divisor() function that allows an application to set
a global divisor that is applied on top of the per-call divisors. Nothing
calls this yet, but loader(8) will be using it to further throttle the
cursor for slow serial consoles.
compressed tarball, aka package. The file system assumes that the
files are layed-out in the same order as needed to allow for the
package to be streamed. As such, it does not read an entire package
into memory first.
Some properties of the file system:
o Files that start with '+' are silently skipped. These are found
in FreeBSD package files.
o Files smaller than or equal to 4KB will be cached in memory and
as such allow for some flexibility in accessing files out of
order.
o Files with the .tgz suffix are assumed to be (sub-)packages and
signal the end for a directory scan.
Obtained from: Juniper Networks, Inc.
to counteract the default behaviour of always trying each and every
file system until one succeeds, or the open fails. The problem with the
loader is that we've implemented features based on this behavior. The
handling of compressed files is a good example of this. However, it is
in general highly undesirable to not have a one-time probe (or taste
in the geom lingo), followed by something similar to a mount whenever
we (first) read from a device. Everytime we go to the same device, we
can reasonably assume it (still) has the same file system. For file
systems that need to do far more that a trivial read of a super block,
not having something similar to a mount operation is disastrous from
a performance (and thus usability) perspective.
But, again, since we've implemented features based on this stateless
approach, things can get complicated quickly if and when we want to
change this. And yet, we sometimes do need stateful behaviour.
For this reason, this change simply introduces exclusive_file_system.
When set to the fsops of the file system to use, the open call will
only try this file system. Setting it to NULL restores the default
behaviour. It's a low-cost (low-brow?) approach to provide enough
control without re-implementing the guts of the loader.
A good example of when this is useful is when we're trying to load
files out of a container (say, a software packaga) that itself lives
on a file system or is fetched over the network. While opening the
container can be done in the normal stateless manner, once it is
opened, subsequent opens should only consider the container.
Obtained from: Juniper Networks, Inc.
The NAND Flash environment consists of several distinct components:
- NAND framework (drivers harness for NAND controllers and NAND chips)
- NAND simulator (NANDsim)
- NAND file system (NAND FS)
- Companion tools and utilities
- Documentation (manual pages)
This work is still experimental. Please use with caution.
Obtained from: Semihalf
Supported by: FreeBSD Foundation, Juniper Networks
In C90, NULL is guaranteed to be declared in <stddef.h> and also in
<string.h>. Though the correct way to define NULL in FreeBSD is to
include <sys/_null.h>, other parts of libstand still require <string.h>
to build; therefore, we keep <string.h> in stand.h and add a note about
this;
- Removing no longer used 'Prototype' definition. Quote from bde@:
'Cruft related to getting incomplete struct declarations within
prototypes forward-declared before the structs. It doesn't mean
"prototype" but only part of a prototype-related hack. No longer
used.'
- Replacing iaddr_t with uintptr_t;
- Removing use of long double to determine alignment. Use a fixed 16 byte
alignment instead;
Reviewed by: bde
Obtained from: DragonFlyBSD (partially)
MFC after: 1 month
called <machine/_types.h>.
o <machine/ansi.h> will continue to live so it can define MD clock
macros, which are only MD because of gratuitous differences between
architectures.
o Change all headers to make use of this. This mainly involves
changing:
#ifdef _BSD_FOO_T_
typedef _BSD_FOO_T_ foo_t;
#undef _BSD_FOO_T_
#endif
to:
#ifndef _FOO_T_DECLARED
typedef __foo_t foo_t;
#define _FOO_T_DECLARED
#endif
Concept by: bde
Reviewed by: jake, obrien
info. This turned out to be rather useful on ia64 for tracking down
malloc/free problems.
Detect duplicate free()'s - otherwise these show up as a guard1 failure
and it looks like corruption instead of something simple like a second
free() where there shouldn't be.
Deal with libz using libc headers and not seeing the malloc/free stuff that
we provide in libstand. Do similar nastiness to what is done for bzlib.
Tested on: i386, ia64 (compile, run)
modules split across several physical medias. Following is how it works:
The splitfs code, when asked to open "foo" looks for a file "foo.split"
which is a text file containing a list of filenames and media names, e.g.
foo.aa "Kernel floppy 1"
foo.ab "Kernel floppy 2"
foo.ac "Kernel and modules floppy"
For each file segment, the process is:
- try to open the file
- prompt "Insert the disk labelled <whatever> and press any key..."
- try to open the file
- return error if file could not be located
RE team is free to use this feature in the upcoming 5.0-DP1.
Reviewed by: msmith, dcs
o In i386's <machine/endian.h>, macros have some advantages over
inlines, so change some inlines to macros.
o In i386's <machine/endian.h>, ungarbage collect word_swap_int()
(previously __uint16_swap_uint32), it has some uses on i386's with
PDP endianness.
Submitted by: bde
o Move a comment up in <machine/endian.h> that was accidentially moved
down a few revisions ago.
o Reenable userland's use of optimized inline-asm versions of
byteorder(3) functions.
o Fix ordering of prototypes vs. redefinition of byteorder(3)
functions, so that the non-GCC (libc asm) case has proper
prototypes.
o Add proper prototypes for byteorder(3) functions in <sys/param.h>.
o Prevent redundant duplicate prototypes by making use of the
_BYTEORDER_PROTOTYPED define.
o Move the bswap16(), bswap32(), bswap64() C functions into MD space
for platforms in which asm versions don't exist. This significantly
reduces the complexity of some things at the cost of duplicate code.
Reviewed by: bde
deprecated in favor of the POSIX-defined lowercase variants.
o Change all occurrences of NTOHL() and associated marcros in the
source tree to use the lowercase function variants.
o Add missing license bits to sparc64's <machine/endian.h>.
Approved by: jake
o Clean up <machine/endian.h> files.
o Remove unused __uint16_swap_uint32() from i386's <machine/endian.h>.
o Remove prototypes for non-existent bswapXX() functions.
o Include <machine/endian.h> in <arpa/inet.h> to define the
POSIX-required ntohl() family of functions.
o Do similar things to expose the ntohl() family in libstand, <netinet/in.h>,
and <sys/param.h>.
o Prepend underscores to the ntohl() family to help deal with
complexities associated with having MD (asm and inline) versions, and
having to prevent exposure of these functions in other headers that
happen to make use of endian-specific defines.
o Create weak aliases to the canonical function name to help deal with
third-party software forgetting to include an appropriate header.
o Remove some now unneeded pollution from <sys/types.h>.
o Add missing <arpa/inet.h> includes in userland.
Tested on: alpha, i386
Reviewed by: bde, jake, tmm
the efficiency of byte-by-byte read operations on filesystems not already
supported by the block cache (especially NFS).
This should be a welcome change for users booting via PXE, as the loader
now reads its startup files almost instantly, instead of taking tens of
seconds.
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.
Fix some ctype problems - isascii() caused a warning if fed an unsigned
char - it's always > 0 and libstand is compiled with -Wall.
Missing prototype/include in printf.c
of getopt (as in, multiple input lines :). This is documented in the
man page and is used in the code, but unistd.h and stand.h do not
declare it. Incidentally, it prevents me fixing a bug in loader's
code... :-)
PR: misc/9373
Submitted by: "Daniel C. Sobral" <dcs@newsguy.com>
etc. associated with the device entry.
Consider EOF an 'error' for fgetstr if we haven't read anything yet.
You *MUST* recompile and reinstall libstand before rebuilding the bootstrap.
disks.
* Fix a whole raft of warnings, printf and otherwise.
* Make zalloc work for alpha (just a case of using the right typedef).
* Add some (disabled) malloc debug printing to stand.h.
compact and much better one donated by Matt Dillon. Implement a simple
sbrk() which uses the existing setheap() api.
Remove the custom allocator from the UFS code. It wasn't working quite
right, and it shouldn't be needed with the new allocator.
Fix a serious problem with changing the value of already-existent
environment variables. Don't attempt to modify the supposedly-const
argument to putenv()
Fix an off-by-one sizing error in the zipfs code detected by the new
allocator.
Submitted by: zmalloc from Matt Dillon <dillon@backplane.com>
