The loader bcache is implementing simple read-ahead to boost the cache.
The bcache is built based on 512B block sizes, and the read ahead is attempting
to read number of cache blocks, based on amount of the free bcache space.
However, there are devices using larger sector sizes than 512B, most obviously
the CD media is based on 2k sectors. This means the read-ahead can not be just
random number of blocks, but we should use value suitable also for use with
larger sectors, as for example, with CD devices, we should read multiple of 2KB.
Since the sector size from disk interface is not too reliable, i guess we can
just use "good enough" value, so the implementation is rounding down the read
ahead block count to be multiple of 16.
This means we have covered sector sizes to 8k.
In addition, the update does implement the end of cache marker, to help to
detect the possible memory corruption - I have not seen it happening so far,
but it does not hurt to have the detection mechanism in place.
Reviewed by: allanjude
Approved by: allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D9179
Need interface to extract information about disk abstraction,
to read disk or partition size depending on the provided argument
and adjust disk size based on information in partition table.
The disk handle from disk_open() has d_offset field to point to
partition start. So we can use this fact to return either whole disk
size or partition size. For this we only need to record partition size
we get from disk_open() anyhow.
In addition, this will also make it possible to adjust the disk media size
based on information from partition table. The problem with disk size is
about some BIOS systems reporting bogus disk size for 2+TB disks, but
since such disks are using GPT partitioning, and GPT does have information
about disk size (alternate LBA + 1), we can use this fact to record disk
size based on partition table.
This patch does exactly this: implements DIOCGSECTORSIZE and DIOCGMEDIASIZE
ioctl, and DIOCGMEDIASIZE will report either disk media size or partition size.
Adds ptable_getsize() call to read partition size in bytes from ptable pointer.
Updates disk_open() to use ptable_getsize() to update mediasize value.
Implements GPT detection function to update ptable size (used by
ptable_getsize()) according to alternate lba (which is location of backup copy
of GPT header table).
Reviewed by: allanjude
Approved by: allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D8594
The disk_* and part_* api is using 64bit values for media size and
offsets. However, the current api is using off_t type, which is signed
64-bit int.
In this context the signed media size does not make any sense, and
the offsets are used to mark absolute, not relative locations.
Also, the data from GPT partition table and some other sources is
already using uint64_t data type, so using signed off_t can cause sign
issues.
Reviewed by: imp
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D8710
after it, which has a conflicting definition of errno. This leads to
the following warning with clang 4.0.0:
In file included from sys/boot/common/reloc_elf32.c:6:
In file included from sys/boot/common/reloc_elf.c:37:
/usr/obj/usr/src/tmp/usr/include/stand.h:155:12: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes]
extern int errno;
^
sys/sys/errno.h:46:26: note: expanded from macro 'errno'
#define errno (* __error())
^
MFC after: 3 days
Apparently the libstand dosfs optimization is a bit too optimistic
and did introduce possible memory corruption.
This patch is backing out the bad part and since this results in
dosfs reading full blocks now, we can also remove extra offset argument
from dv_strategy callback.
The analysis of the issue and the backout patch is provided by Mikhail Kupchik.
PR: 214423
Submitted by: Mikhail Kupchik
Reported by: Mikhail Kupchik
Reviewed by: bapt, allanjude
Approved by: allanjude (mentor)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8644
When WITH_REPRODUCIBLE_BUILD=yes is set in src.conf(5), eliminate the
time, user, and host from the loader's version information. This allows
builds to produce bit-for-bit identical output.
Reviewed by: bapt
MFC after: 1 month
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8842
The Makefile rule to create vers.c for loader version info was
previously duplicated in each of the various loader Makefiles.
Instead, share a common rule in Makefile.inc.
Reviewed by: bapt
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8841
Instead of repeating "%s, Revision %s" "(%s %s)" in each loader, just
create the full version string in vers.c
Reviewed by: bapt
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8823
Prior to this change the loader self relocation code interpreted amd64's
rela relocations as if they were rel relocations, discarding the addend.
This "works" because GNU ld 2.17.50 stores the addend value in both the
r_addend field of the relocation (as expected) and at the target of the
relocation.
Other linkers, and possibly other versions of GNU ld, won't have this
behaviour, so interpret the relocations correctly.
Reported by: George Rimar
Reviewed by: andrew
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8681
lsdev command does walk over devsw list, prints list element name and
will use dv_print() callback to print the device list.
Unfortunately this approach will add unneeded noise when there are no
particular devices detected.
To remove "empty" device section headers, the dv_print() callback
should print the header instead.
In addition, fixed dv_print callback for md module.
Reviewed by: imp
Approved by: imp (mentor)
Differential Revision: https://reviews.freebsd.org/D8551
/loader.rc) to specify a Forth file to read from the pkgfs tarball and
process by Ficl.
This allows for the tarball to do runtime things like load a
platform-specific FDT blob, among other things.
Reviewed by: imp
Approved by: sjg (mentor)
MFC after: 2 weeks
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D8494
This change does modify devsw dv_print() to return the int value,
enabling walkers to interrupt the walk on non zero value from dv_print().
This will allow the pager_print actually to stop displaying data on
user input, and additionally pager is used in various *dev_print callbacks,
where it was missing.
For test, lsdev [-v] command should display data by screenfuls and should
stop when the key 'q' is pressed on pager prompt.
Reviewed by: allanjude
Approved by: allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D5461
FICL definitions not in ficl/ficl32 files broke this generally. This
makes that stuff conditional on BOOT_FORTH. Also, move definitions
related to the architecture (FICL_CPUARCH and friends) into
Makefile.ficl that all parts of the tree that include files with ficl
need to include (but only if MK_FORTH == yes). In addition, had to fix
library ordering issue with LIBSTAND to keep it last. Without boot
forth, there's no references to memset to bring in memset.o from
libstand.a to satisfy libgeliboot.a's use of it. Listing libstand last
solves this issue (and it's the proper place for libstand to boot).
functions to call at the appropriate time to register new forth
words. In the past we've done this with ifdef soup, but now if the
file is included in the build, we'll get the new forth words.
Use this new functionality to move the pci bios stuff out of loader.c
by moving it to biospci.c.
Move the pnp functionality to common/pnp.c.
Move the inb/outb forth words to the i386 sysdep.c file where their
implementation is defined.
Adjust the efi linker scripts and build machinery to cope.
his should be an invisible change to forth scripts and user
experience.
Differential Revision: https://reviews.freebsd.org/D8145
than to store the location of a forth word that is subsequently never
used. It was last used before the 2.03 ficl upgrade in r51786. It was
only used from r43614 (so Feb-Sept 1999) on head and in the 3.x branch
(merged r43715 3.1 -> EOL). Remove it since nobody cared enough to
report the bug in the last 18 years rather than fix it. It's need
seems to have passed in the 2.03 ficl update.
Differential Revision: https://reviews.freebsd.org/D8150
replaced by /boot/loader.rc for 3.1 (r42682). In May 2000, this was
documented as deprecated (r61942) (between FreeBSD 4.0 and
4.1). Remove it since it's not been the preferred method in 17 years
and has been deprecated for 16.
Differential Revision: https://reviews.freebsd.org/D8142
dev_net.c code.
The NETIF_OPEN_CLOSE_ONCE flag was added in r201932 to prevent that behaviour
on some architectures (sparc64 and powerpc64) the default was left to always
open and close the device for each open and close of a file by the loader
because it was necessary for u-boot on arm.
Since it has been added, the flag was turned on for every arches including the
u-boot loader for arm.
This also fixes netbooting on RPi3 (tested by gonzo@)
For the loader.efi it greatly speeds up netbooting
Reviewed by: emaste, gonzo, tsoome
Approved by: gonzo
MFC after: 1 month
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D8230
R_AARCH64_RELATIVE relocation found on arm64. It would try to add the
contents of the memory location being relocated to the base address and
the relocation addend. This worked when the contents was zero, however
this now seems to be set to the value of the addend so we add this twice.
Fix this by just setting the memory to the computed value.
Reviewed by: emaste
MFC after: 1 week
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D8177
The command interpreter does leave command_errmsg as is after printing its
content, assuming the next command will reset it in bf_command(). However,
in case the forth native word is defined as builtin, the bf_command is not
used and forth words will also end up the command_errmsg content printed.
Since command_errmsg is pointer to actual error message, which can be static
read only string, we can not just set *command_errmsg = '\0', instead we need
to reset the pointer itself.
Illumos issue: https://www.illumos.org/issues/7405
Reported by: Igor Kozhukhov.
Reviewed by: allanjude
Approved by: allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D8032
Add a new 'netproto' variable which can be set for now to
NET_TFTP or NET_NFS (default to NET_NONE)
From the dhcp options if one sets the root-path option to:
"ip:path", the loader will consider it is booting over NFS
(meaning same behaviour as the default current behaviour)
if the dhcp option "tftp server address" is set (option 150)
the loader will consider it is booting over tftpfs, it will then
consider the root-path options with 2 possible case
1. "path" then the IP of the tftp server will be the one passed by
the option 150, and the files will be retrieved under "path" on the tftp
server
2. "ip:path" then the IP of the tftp server will be the one passed in
the option "overwritting the IP from the option 150.
We could not "abuse" the rootpath option in the form or tftp://ip:path because
this is already used for other purpose by iPXE preventing any chainload from
iPXE to the FreeBSD loader.
Given at each open(), the loader loops over all available filesystems and keep
the "best" error, we needed to prevent tftpfs to fallback on nfs and vice versa.
the tftpfs and nfs implementation in libstand now return EINVAL early if
'netproto' for that purpose.
Reviewed by: tsoome
Relnotes: yes
Sponsored by: Gandi.net
Differential Revision: https://reviews.freebsd.org/D7628
Allow netbooting on efi without having to setup any NFS server by rebuilding the
loader with LOADER_TFTP_SUPPORT like for the i386 pxeloader
Sponsored by: Gandi.net
strings provided by user/config files. This update is replacing sprintf with
snprintf for cases the command_errbuf is built from dynamic content.
PR: 211958
Reported by: ecturt@gmail.com
Reviewed by: imp, allanjude
Approved by: imp (mentor), allanjude (mentor)
Differential Revision: https://reviews.freebsd.org/D7563
parse() is the boot loader's interp_parse.c is too naive about quotes
both single and double quotes were allowed to be mixed, and single
quotes did not follow the usual semantics (re variable expansion).
The old code did not check for terminating quotes
This update implements:
* distinguishing single and double quote
* variable expansion will not be done inside single quote protected area
* will preserve inner quote for values like "value 'some list'"
* ending quote check.
this diff does not implement ending quote order check, it shouldn't
be too hard, needs some improvements on parser state machine.
PR: 204602
Submitted by: Toomas Soome <tsoome@me.com>
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D6000
dosfs (fat file systems) can perform reads of partial sectors
bcache should support such reads.
Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D6475
return value when it could return 1 (indicating we should stop).
Fix a few instances of pager_open() / pager_close() not being called.
Actually use these routines for the environment variable printing code
I just committed.
The new bcache code does not know the size of the disk, and therefore may attempt to read past the end of the disk while trying to fill its read-ahead cache.
This is usually not an issue, it fails gracefully on all of my machines, but some BIOSes seem to retry the reads for up to 30 seconds each, resulting in a long stall during boot
Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: jhb, np
Differential Revision: https://reviews.freebsd.org/D6109
rounddown2 tends to produce longer lines than the original code
and when the code has a high indentation level it was not really
advantageous to do the replacement.
This tries to strike a balance between readability using the macros
and flexibility of having the expressions, so not everything is
converted.
The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.
Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.
Submitted by: Toomas Soome <tsoome@me.com>
Reviewed by: delphij (previous version), emaste (previous version)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D4713
to the boot.netif.mtu env var, which will be picked up by pre-existing code
in nfs_mountroot() and used to configure the interface accordingly.
This should bring the same functionality when the bootp/dhcp work is done
by loader(8) as r297150 does for the in-kernel BOOTP case.
the boot loader should not skip over these anymore while loading images.
Otherwise the kernel can still panic when it doesn't find the .eh_frame
section belonging to the .rela.eh_frame section.
Unfortunately this will require installing boot loaders from sys/boot
before attempting to boot with a new kernel.
Reviewed by: kib
MFC after: 2 weeks
X-MFC-With: r296419
