Commit Graph

409 Commits

Author SHA1 Message Date
Toomas Soome
3949295f36 libstand/dosfs: cache FAT32 in 128 Kb blocks to save loader memory
Current implementation of dosfs in libstand reads full File Allocation Table to
the RAM in the initialization code. In the extreme case of FAT32 filesystem,
this structure will take up to 256-1024 Mb of loader memory, depending on the
cluster size.

Proposed patch reduces libstands/dosfs memory requirements to 128 Kb for all
variants of dosfs filesystem. For FAT12 and FAT16 filesystems, File Allocation
Table is cached in full, as before. For FAT32, File Allocation Table is broken
into the equal blocks of 128 Kilobytes (32768 entries), and only current block
is cached.

Because per-filesystem context is now small, global FAT cache (for all
instances of dosfs filesystem) is replaced by local per-instance cache.

Submitted by:	Mikhail.Kupchik_gmail.com
Reviewed by:	tsoome, allanjude
Differential Revision:	https://reviews.freebsd.org/D9547
2017-04-06 20:12:39 +00:00
Toomas Soome
ffd08eb064 loader: want mechanism to avoid RA with bcache
While we have mechanisms in place to protect ourselves against the read
behind the disk end, there is still one corner case. As the GPT
partition table has backup table at the end of the disk, and we yet
do not know the size of the disk (if the wrong size is provided by the
firmware/bios), we need to limit the reads to avoid read ahead in such case.

Note: this update does add constant into stand.h, so the incremental build
will need to get local stand.h updated first.

Reviewed by:	allanjude
Differential Revision:	https://reviews.freebsd.org/D10187
2017-04-06 15:57:53 +00:00
Allan Jude
9f67bd210a Add explicit_bzero() to libstand, and switch GELIBoot to using it
Make sure sensitive memory is properly cleared when finished with it

Reviewed by:	Eric McCorkle <eric@metricspace.net>
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D9798
2017-03-31 00:04:32 +00:00
Toomas Soome
cb83812c70 Remove OLD_NFSV2 from loader and libstand
We have parallel NFSv2 and NFSv3 reader implementations, only configurable at
build time, defaulting to v3. Remove v2.

Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D10206
2017-03-30 19:32:25 +00:00
Toomas Soome
c1e968fb62 loader: verify the value from dhcp.interface-mtu and use snprintf to set mtu
Since the uset can set dhcp.interface-mtu, we need to try to validate the
value. So we verify if the conversion to int is successful and we will not
allow to set value greater than max IPv4 packet size.

Also use snprintf for safety.

Reviewed by:	allanjude, bapt
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8492
2017-03-20 22:20:17 +00:00
Toomas Soome
7404367b95 libstand: verify value provided by nfs.read_size
Implement simple value check and feedback.

Reviewed by:	allanjude, rpokala
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8491
2017-03-20 19:16:28 +00:00
Mariusz Zaborski
4cd385d9e9 Try to extract the RFC1048 data from PXE. If we get enough info we can skip
the bootp(). It removes unnecessary DHCP request from pxeloader.

Submitted by:	kczekirda
Sponsored by:	Oktawave
Initiated by:	Matthew Dillon
Reviewed by:	smh, gnn, bapt, oshogbo
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D9847
2017-03-09 06:01:24 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Ed Maste
1dc349ab95 prefix UFS symbols with UFS_ to reduce namespace pollution
Specifically:
  ROOTINO -> UFS_ROOTINO
  WINO -> UFS_WINO
  NXADDR -> UFS_NXADDR
  NDADDR -> UFS_NDADDR
  NIADDR -> UFS_NIADDR
  MAXSYMLINKLEN_UFS[12] -> UFS[12]_MAXSYMLINKLEN (for consistency)

Also prefix ext2's and nandfs's NDADDR and NIADDR with EXT2_ and NANDFS_

Reviewed by:	kib, mckusick
Obtained from:	NetBSD
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D9536
2017-02-15 19:50:26 +00:00
Conrad Meyer
0ecf59f68f ufs: Use UFS_MAXNAMLEN constant
(like NFS, EXT2FS, SVR4, IBCS2) instead of redefining the MAXNAMLEN
constant.

No functional change.

Reviewed by:	kib@, markj@
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D9500
2017-02-09 17:47:01 +00:00
Toomas Soome
467c82cb84 loader: Replace EFI part devices.
Rewrite EFI part device interface to present disk devices in more
user friendly way.

We keep list of three types of devices: floppy, cd and disk, the
visible names: fdX: cdX: and diskX:

Use common/disk.c and common/part.c interfaces to manage the
partitioning.

The lsdev -l will additionally list the device path.

Reviewed by:	imp, allanjude
Approved by:	imp (mentor), allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8581
2017-02-06 09:18:47 +00:00
Baptiste Daroussin
0922cf8d32 remove network mask calculation for Classful network
Nowadays it's not necessary to compute network mask from the IP address and
compare to given by DHCP.

Submitted by:	kczekirda
Reviewed by:	glebius, bapt
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D8740
2017-01-07 23:42:17 +00:00
Xin LI
e16f0839c6 Restructure libz, place vendor files in contrib/zlib like other third
party software, this provides more standarized import workflow and
makes future upgrades easier.

The following files are new with this commit:

	zconf.h.in
	zlib.map
	zlib.pc.in

They are not connected to build, but were kept in tree for reference
for future maintenance.

All our local trivial changes were applied to contrib/zlib, and the
contrib/zlib vendor source code is intended to 100% match lib/libz
before this commit.

MFC after:	2 weeks
2017-01-04 09:30:47 +00:00
Toomas Soome
ba8cf3277a loader: nandfs calls strategy with one extra argument.
The strategy argument cleanup in r310850 did miss another call to strategy(),
and left it with extra argument.

Reviewed by:	allanjude
Approved by:	allanjude (mentor)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D9003
2016-12-30 20:00:41 +00:00
Toomas Soome
1ecc859193 dosfs support in libstand is broken since r298230
Apparently the libstand dosfs optimization is a bit too optimistic
and did introduce possible memory corruption.

This patch is backing out the bad part and since this results in
dosfs reading full blocks now, we can also remove extra offset argument
from dv_strategy callback.

The analysis of the issue and the backout patch is provided by Mikhail Kupchik.

PR:		214423
Submitted by:	Mikhail Kupchik
Reported by:	Mikhail Kupchik
Reviewed by:	bapt, allanjude
Approved by:	allanjude (mentor)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D8644
2016-12-30 19:06:29 +00:00
Toomas Soome
40a969525a libstand: dosfs cstyle cleanup for return keyword.
Reviewed by:	bapt, imp, allanjude
Approved by:	imp (mentor), allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8678
2016-12-01 19:19:20 +00:00
Toomas Soome
cbd6713146 Loader paged/pageable data is not always paged.
This change does modify devsw dv_print() to return the int value,
enabling walkers to interrupt the walk on non zero value from dv_print().

This will allow the pager_print actually to stop displaying data on
user input, and additionally pager is used in various *dev_print callbacks,
where it was missing.

For test, lsdev [-v] command should display data by screenfuls and should
stop when the key 'q' is pressed on pager prompt.

Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D5461
2016-11-08 06:50:18 +00:00
Baptiste Daroussin
ad9f2cecd7 When issuing a PXE dhcp request, always issue a param request (DHCP option 55)
with all dhcp parameters we might be interested in.

Some DHCP server like the new kea (by ISC) expect it.

This makes pxeboot functional with ISC kea.

Submitted by:	Vincent Legout <vincent.legout@gandi.net>
MFC after:	1 month
Sponsored by:	Gandi.net
2016-10-31 15:11:55 +00:00
Toomas Soome
ef97c45728 Fix remaining bugs in libstancd/cd9660.c reported by Bruce Evans.
Fixed text in comments.

Reported by:	Bruce Evans <brde@optusnet.com.au>
Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8119
2016-10-03 14:07:15 +00:00
Toomas Soome
062b8022ac Fix remaining cstyle issues in libstand/cd9660.c
Reported by:	Bruce Evans <brde@optusnet.com.au>
Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8107
2016-10-01 07:46:28 +00:00
Toomas Soome
6a21906bea cstyle fix of cd9660_open in libstand
rS306534 did create bad cstyle by my mistake, correcting it.

Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8103
2016-09-30 23:19:08 +00:00
Toomas Soome
339e076e67 cd9660_open should check for padding
The libstand cd9660 reader code must check for extents having padding.

Reviewed by:	jhb
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D8101
2016-09-30 22:28:20 +00:00
Baptiste Daroussin
d600e95429 Add a comment on the #endif related to the guards
Reported by:	tsoome
2016-09-13 15:41:05 +00:00
Sepherosa Ziehau
ff9eac2e6d pxeboot: Add nfs.read_size tunable.
Increase this tunable improves kernel loading speed.

Submitted by:	Jun Su <junsu microsoft com>
Reviewed by:	rpokala, wblock (previous version)
Obtained from:	DragonFlyBSD
MFC after:	1 week
Sponsored by:	Microsoft
Differential Revision:	https://reviews.freebsd.org/D7756
2016-09-08 09:11:13 +00:00
Warner Losh
949b56ba66 Renumber the advertising clause. 2016-09-06 15:17:35 +00:00
Baptiste Daroussin
6a695123b0 Netboot: allow both tftpfs and nfs in both pxeboot and loader.efi
Add a new 'netproto' variable which can be set for now to
NET_TFTP or NET_NFS (default to NET_NONE)

From the dhcp options if one sets the root-path option to:
"ip:path", the loader will consider it is booting over NFS
(meaning same behaviour as the default current behaviour)

if the dhcp option "tftp server address" is set (option 150)
the loader will consider it is booting over tftpfs, it will then
consider the root-path options with 2 possible case
1. "path" then the IP of the tftp server will be the one passed by
the option 150, and the files will be retrieved under "path" on the tftp
server
2. "ip:path" then the IP of the tftp server will be the one passed in
the option "overwritting the IP from the option 150.

We could not "abuse" the rootpath option in the form or tftp://ip:path because
this is already used for other purpose by iPXE preventing any chainload from
iPXE to the FreeBSD loader.

Given at each open(), the loader loops over all available filesystems and keep
the "best" error, we needed to prevent tftpfs to fallback on nfs and vice versa.
the tftpfs and nfs implementation in libstand now return EINVAL early if
'netproto' for that purpose.

Reviewed by:	tsoome
Relnotes:	yes
Sponsored by:	Gandi.net
Differential Revision:	https://reviews.freebsd.org/D7628
2016-08-31 13:16:40 +00:00
Toomas Soome
9edcae95e4 recvtftp() is broken for large files, report file size
The tftp download for large files will cause internal block id
to wrap to 0 as the data type is unsigned short.

Also provide file size information for stat.

PR:		200500
Reported by:	tsoome
Reviewed by:	allanjude
Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D7660
2016-08-31 09:23:09 +00:00
Toomas Soome
41638b75ce Unused variables and cstyle fix for loader dosfs
Reviewed by:	imp, allanjude
Approved by:	imp (mentor), allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D7659
2016-08-26 14:58:57 +00:00
Warner Losh
de06886d2b Fix a bug in the parsing code: always use the len and not 8. 2016-05-20 05:33:26 +00:00
Warner Losh
da019dfdd5 Add support for %S to libstand as well so /boot/loader and friends can
use it.
2016-05-17 21:23:13 +00:00
Pedro F. Giffuni
ab881fadba libstand: make more use of the howmany() macro when available.
We have a howmany() macro in the <sys/param.h> header that is
convenient to re-use as it makes things easier to read.
2016-04-26 01:19:36 +00:00
Allan Jude
7509c09b1b Unbreak the build if you enable WITH_NAND
Followup to r298230

Submitted by:	Nikolai Lifanov <lifanov@mail.lifanov.com> (original version)
Sponsored by:	ScaleEngine Inc.
2016-04-19 02:06:02 +00:00
Allan Jude
87ed2b7f5a A new implementation of the loader block cache
The block cache implementation in loader has proven to be almost useless, and in worst case even slowing down the disk reads due to insufficient cache size and extra memory copy.
Also the current cache implementation does not cache reads from CDs, or work with zfs built on top of multiple disks.
Instead of an LRU, this code uses a simple hash (O(1) read from cache), and instead of a single global cache, a separate cache per block device.
The cache also implements limited read-ahead to increase performance.
To simplify read ahead management, the read ahead will not wrap over bcache end, so in worst case, single block physical read will be performed to fill the last block in bcache.

Booting from a virtual CD over IPMI:
0ms latency, before: 27 second, after: 7 seconds
60ms latency, before: over 12 minutes, after: under 5 minutes.

Submitted by:	Toomas Soome <tsoome@me.com>
Reviewed by:	delphij (previous version), emaste (previous version)
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D4713
2016-04-18 23:09:22 +00:00
Pedro F. Giffuni
d5808ebf65 libstand: use NULL instead of zero for pointers. 2016-04-18 14:45:56 +00:00
Glen Barber
d60840138f MFH
Sponsored by:	The FreeBSD Foundation
2016-04-04 23:55:32 +00:00
Bjoern A. Zeeb
936ce65444 Correct comment.
MFC after:	2 weeks
2016-03-22 14:08:00 +00:00
Ian Lepore
00f66a5236 If the dhcp server delivers an interface-mtu option, parse it and store
the value in a new global intf_mtu for use by the application.

These changes were inspired by the patch provided by Robert Blayzor in
PR 187094, and will allow loader(8) to propagate the value to the kernel
along with the other nfs_diskless parms delivered via environment vars.
2016-03-21 14:58:12 +00:00
Ian Lepore
678849441d Actually garbage collect the unused code, as mentioned in r297147, which
this change should have been part of.
2016-03-21 14:39:03 +00:00
Ian Lepore
3cf266f1bd Garbage collect the bswap routines from libstand. The declaration was
wrapped in an i386 ifdef with a comment questioning their usefulness even
there.  It turns out they aren't referenced anywhere, but their presence
prevents using sys/endian.h in libstand code.

These days, sys/endian.h provides much better support for such things, using
compiler builtins and inline functions (and creating connections between
libstand code and header files from sys/ would not be breaking new ground).
2016-03-21 14:21:32 +00:00
Glen Barber
b655ec9752 MFH
Sponsored by:	The FreeBSD Foundation
2016-03-06 04:13:17 +00:00
Glen Barber
52259a98ad MFH
Sponsored by:	The FreeBSD Foundation
2016-03-02 16:14:46 +00:00
Dimitry Andric
14e9c9161c Merge ^/head r295902 through r296006. 2016-02-24 21:38:51 +00:00
Bryan Drewery
bd18fd57db DIRDEPS_BUILD: Regenerate without local dependencies.
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.

Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:11 +00:00
Glen Barber
a70cba9582 First pass through library packaging.
Sponsored by:	The FreeBSD Foundation
2016-02-04 21:16:35 +00:00
Dimitry Andric
d9b9dae1a9 Merge ^/head r294169 through r294598. 2016-01-22 20:41:56 +00:00
Ruslan Bukin
0bfee92849 Bring in initial libc and libstand support for RISC-V.
Reviewed by:	andrew, emaste, kib
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D4943
2016-01-17 15:21:23 +00:00
Dimitry Andric
89d3f0ea4e Merge ^/head r293430 through r293685. 2016-01-11 19:36:44 +00:00
Andrey V. Elsukov
6299675b9f Make tohex() work as expected. 2016-01-09 08:04:29 +00:00
Andrey V. Elsukov
06ddd0b7ac Fix a typo.
PR:		205722
2016-01-09 08:02:35 +00:00
Dimitry Andric
b229c1a0b0 Merge ^/head r293280 through r293429. 2016-01-08 17:42:14 +00:00