242861 Commits

Author SHA1 Message Date
bdrewery
dd9392fbd8 packages: Allow actually building individual world packages in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:54 +00:00
bdrewery
59c2de28ee packages: Allow creating kernel/world packages in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:51 +00:00
bdrewery
93ad18e8c3 packages: Allow staging world/kernel in parallel.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:47 +00:00
bdrewery
1ed4ce3467 packages: Allow stageworld/stagekernel to run with make jobs.
The -B was originally added in projects/release-pkg r289381 as a copy
of what 'make world' did at the time.  The -B was removed from
the 'installworld' call in 'world' in r303844 though.  The staging
of files is safe to run in parallel.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 18:26:33 +00:00
imp
25c1ae6892 Decode FreeBSD 11 compat stat, fstat and lstat calls. 2017-06-23 18:06:46 +00:00
imp
6563dfe551 Forward compatibility for ino64.
Add forward compatibility so that new binaries can run on old
kernels. If the new system call from ino64 isn't available on your
system, then the old one will be used and the results translated.  The
stat and statfs families of functions are fully emulated. While not
required by policy, in this case it is helpful to our users to provide
this compatibility. In this case, it allows rollback of the kernel
after installing a new userland should a problem be discovered. It
also prevents foot-shooting if a user does an install before rebooting
with the new kernel. Finally, it allows the use case where one needs
to run new binaries on an old kernel as part of an upgrade process.

The getdirentries family uses tricks that may not work on remote
filesystems. Specifically, it uses a buffer 1/4 the size requested to
get the data from he old syscall.

The code carefully uses direct syscalls for old system calls to avoid
referencing freebsd11_* symbols, which contaminate ld-elf.so.1's
export table due to its use of stat functions, which causes errno to
be incorrect in client programs due to the wrong *stat* function being
resolved in some cases.

This code should removed sometime after 12 is branched.

Tested on: 12-current binaries on a 10.3-beta kernel run and return
       consistent results. 12-current kernel and userland with
       packages from before ino64 was committed also work.

Differential Revision: https://reviews.freebsd.org/D11185
Reviewed by: kib@, emaste@
2017-06-23 18:06:20 +00:00
sobomax
8f9a541591 Don't leak file descriptor in some cases.
Reported by:	cem
MFC after:	6 weeks
2017-06-23 17:39:00 +00:00
manu
54e47e4fc2 ipfw: Note that bandwidth can take G suffix in the manpage
Reported by:	Jose Luis Duran (github)
2017-06-23 17:31:07 +00:00
emaste
3bde37655a Introduce LINKER_FEATURES to avoid duplicating version logic
Submitted by:	bdrewery
Reported by:	kib
2017-06-23 17:21:37 +00:00
bdrewery
14e1726829 Set compiler metadata for stageworld/distributeworld.
This fixes LD errors during 'make packages' but also for the unlikely case of
'buildworld' on 1 system and 'packages' on another [1].

PR:		212877 [1]
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 16:38:46 +00:00
bdrewery
e92b0ec4e0 Allow ALWAYS_BOOTSTRAP_MAKE to force bmake bootstrapping.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-23 16:38:18 +00:00
emaste
315027dfef enable --build-id for the kernel link
A Build-ID is an identifier generated at link time to uniquely identify
ELF binaries.  It allows efficient confirmation that an executable or
shared library and a corresponding standalone debuginfo file match.
(Otherwise, a checksum of the debuginfo file must be calculated when
opening it in a debugger.)

The FreeBSD base system includes GNU bfd ld 2.17.50 as the linker for
architectures other than arm64.  Build-ID support was added to bfd ld
shortly after that version, so was not previously available to us.

We can now start making use of Build-ID as we migrate to using lld or
bfd ld from ports, conditionally enabled based on the LINKER_TYPE and
LINKER_VERSION make variables added in r320244 and subsequent commits.

Reviewed by:	dim
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11314
2017-06-23 15:57:58 +00:00
bapt
032c91cd1d sesutil no longer depends on libsbuf
Sponsored by:	Gandi.net
2017-06-23 15:27:23 +00:00
bapt
5dd5b83dbd Directly print the extra status instead of filling a buffer
then printing it.

This prepares the code to make it libxo friendly

Reviewed by:	manu, Nikita Kozlov (nikita elyzion.net)
MFC after:	2 weeks
Sponsored by:	Gandi.net
2017-06-23 15:09:08 +00:00
asomers
b19d13f1dc style fixes in bin/echo/tests
Submitted by:	shivansh
Reviewed by:	asomers
MFC after:	2 weeks
X-MFC-With:	319626
Sponsored by:	Google, Inc (GSoC 2017)
Differential Revision:	https://reviews.freebsd.org/D11318
2017-06-23 15:02:49 +00:00
manu
72d95f3521 ipfw: dummynet: Add 'G' and 'g' suffix for bandwidth configuration/display
MFC after:	2 weeks
Sponsored by:	Gandi.net
2017-06-23 14:00:28 +00:00
bapt
5ba83e707d Do not use sprintf(3) when not needed, while here,
prefer snprintf(3) over sprintf(3)
2017-06-23 13:26:13 +00:00
avg
d0705fcb11 jedec_ts: add support for devices manufactured by IDT
Full manufacturer name is Integrated Device Technology, Inc.
Supported devices include TSE2002B3C and TS3000B3A.

MFC after:	1 week
2017-06-23 11:55:43 +00:00
mmokhi
889f8689d9 Fix caveat in new implementation of linprocfs_docpuinfo():
Prevent kernel panic in case that extended-cpuid isn't supported by CPU

Reviewed by:	kib, ngie, trasz
Approved by:	trasz
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D11294
2017-06-23 10:36:27 +00:00
tuexen
b7801f570d Check the length of a COOKIE chunk before accessing fields in it.
Thanks to Felix Weinrank for reporting the issue he found by using
libFuzzer.

MFC after:	3 days
2017-06-23 10:09:49 +00:00
tuexen
2abe6d833d Use a longer buffer for messages in ERROR chunks.
This allows them to be sent in a non truncated way and addresses a warning
given by newver versions of gcc.
Thanks to Anselm Jonas Scholl for reporting it and providing a patch.
2017-06-23 09:27:31 +00:00
avg
c5a3af7ebe fix gcc-specific fallout from r320156, MFV of r318946, ZFS ABD
Reported by:	jhibbits
MFC after:	1 week
X-MFC with:	r320156
2017-06-23 08:42:53 +00:00
tuexen
4b087d3335 Honor the backlog field. 2017-06-23 08:35:54 +00:00
tuexen
fc06311c80 Improve compilation on platforms different from FreeBSD. 2017-06-23 08:34:01 +00:00
avg
2da4ac8256 jedec_ts: read device id from the correct register
Due to my braino / typo the driver was reading the Vendor ID register
twice.

MFC after:	3 days
2017-06-23 06:25:39 +00:00
emaste
5816cc6184 change GNU ld LINKER_TYPE from binutils to bfd
GNU binutils includes two linkers: ld.bfd and ld.gold.  For clarity use
LINKER_TYPE=bfd to refer to ld.bfd, the original binutils linker that
identifies itself as "GNU ld".

Discussed with:	bdrewery
2017-06-23 03:55:00 +00:00
cy
781dbf5aab Replace AF_INET6 ifdefs with USE_INET6 ifdefs. This is more consistent
and guaranteed to build everywhere in ipfilter.

Not all of this commit can be MFCed. Some is original code while others
are not.
2017-06-23 02:42:04 +00:00
cy
663fd2f279 Replace the leap-seconds file in r320242 from USNO -
ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.3701462400 - with a
leap-seconds file from NIST at ftp://time.nist.gov/pub/. The USNO
version of the file changes the last documented leap second update
time whereas the NIST version does not. The expiration of the USNO
version of the file is also one month short.

Requested by:	ian@
Obtained from:	ftp://time.nist.gov/pub/leap-seconds.3676924800
MFC after:	3 days
2017-06-23 01:05:49 +00:00
asomers
538320ee5d Clarify usage of aio(4) with kqueue(2)
Reviewed by:	jhb
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D11299
2017-06-23 00:40:09 +00:00
gjb
75e234a52c Fix, for the fourth or fifth time now, looking up where boot1.efi
is located.

MFC after:	5 days
X-MFC-Note:	maybe
X-MFC-With:	r320252, r320253
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:27:40 +00:00
gjb
8d06963e5e Remove two configuration entries used for debugging that snuck in.
MFC after:	5 days
X-MFC-Note:	maybe
X-MFC-With:	r320252
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:11:34 +00:00
gjb
47a8c8353f In release/release.sh:
- Rename chroot_arm_armv6_build_release() to chroot_arm_build_release()
  and make it hardware agnostic (such as armv6 -vs- armv7 -vs- arm64).

- Evaluate EMBEDDED_TARGET differently so release/tools/arm.subr can
  be used for arm/armv6 and arm64/aarch64.

- Update comments and copyright.

In release/tools/arm.subr:
- In arm_create_disk(), change the default alignment from 63 to 512k,
  fixing a boot issue on arm64 and EFI. [1]

- Update comments and copyright.

Add a RPI3 configuration file, pieces obtained from Crochet.

Obtained from:	Crochet [1]
MFC after:	5 days
X-MFC-Note:	maybe
Sponsored by:	The FreeBSD Foundation
2017-06-23 00:08:36 +00:00
bdrewery
656ec7cc2c Provide proper values for X_LINKER_TYPE/VERSION when XLD == LD.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:10 +00:00
bdrewery
3132dadd89 Don't overwrite already-set LINKER_VERSION/LINKER_TYPE.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 22:53:07 +00:00
bdrewery
fe8fd0e30d LINKER_VERSION: Support external binutils.
The ports binutils stores the version in the 5th word so just look for
a version using a pattern instead.

Reported by:	rpokala
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-22 21:22:12 +00:00
bdrewery
1af3505e35 Pass along LINKER_* vars during installworld and show in test-system-compiler.
Sponsored by:	Dell EMC Isilon
2017-06-22 21:03:30 +00:00
bdrewery
74c01096d1 Support cached linker values in environment.
This is similar to r289659 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11310
2017-06-22 21:03:27 +00:00
bdrewery
d53b98880f Support XLD for setting X_LINKER_TYPE and X_LINKER_VERSION.
This is similar to r300350 for bsd.compiler.mk.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11309
2017-06-22 21:03:24 +00:00
bdrewery
7cf0e0ad69 Add basic bsd.linker.mk auto included from bsd.compiler.mk.
This will provide LINKER_TYPE and LINKER_VERSION.

MFC after:	2 weeks
Reviewed by:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11308
2017-06-22 21:03:20 +00:00
bdrewery
4a42972a1d Fix spelling error.
Reported by:	arc
Sponsored by:	Dell EMC Isilon
2017-06-22 20:32:23 +00:00
cy
91be7aa5ee Update leap-seconds to leap-seconds.3701462400.
As per https://datacenter.iers.org/eop/-/somos/5Rgv/latest/16:

     INTERNATIONAL EARTH ROTATION AND REFERENCE SYSTEMS SERVICE (IERS)

SERVICE INTERNATIONAL DE LA ROTATION TERRESTRE ET DES SYSTEMES DE REFERENCE

SERVICE DE LA ROTATION TERRESTRE
OBSERVATOIRE DE PARIS
61, Av. de l'Observatoire 75014 PARIS (France)
Tel.      : 33 (0) 1 40 51 23 35
FAX       : 33 (0) 1 40 51 22 91
Internet  : services.iers@obspm.fr

                                             Paris, 9 January 2017

                                             Bulletin C 53

                                             To authorities responsible
                                             for the measurement and
                                             distribution of time

                          INFORMATION ON UTC - TAI

 NO leap second will be introduced at the end of June 2017.
 The difference between Coordinated Universal Time UTC and the
 International Atomic Time TAI is :

     from 2017 January 1, 0h UTC, until further notice : UTC-TAI = -37 s

 Leap seconds can be introduced in UTC at the end of the months of December
 or June,  depending on the evolution of UT1-TAI. Bulletin C is mailed every
 six months, either to announce a time step in UTC, or to confirm that there
 will be no time step at the next possible date.

                                            Christian BIZOUARD
                                            Director
                                            Earth Orientation Center of IERS
					    Observatoire de Paris, France

Obtained from:	ftp://tycho.usno.navy.mil/pub/ntp/leap-seconds.3701462400
MFC after:	3 days
2017-06-22 19:25:17 +00:00
ed
49378fdc8c Use __ISO_C_VISIBLE, as opposed to testing __STDC_VERSION__.
FreeBSD's C library uses __STDC_VERSION__ to determine whether the
compiler provides language features specific to a certain version of the
C standard. __ISO_C_VISIBLE is used to specify which library features
need to be exposed.

max_align_t currently uses __STDC_VERSION__, even though it should be
using __ISO_C_VISIBLE to remain consistent with the rest of the headers
in include/.

Reviewed by:	dim
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D11303
2017-06-22 18:39:52 +00:00
avg
c93bff419b MFV r319950: 5220 L2ARC does not support devices that do not provide 512B access
FreeBSD note: the actual change has been in FreeBSD since r297848.  This
commit accounts for integration of that change with subsequent changes,
especially r320156 (MFV of r318946) and r314274.

illumos/illumos-gate@403a8da73c
403a8da73c

https://www.illumos.org/issues/5220
  There are disk devices that have logical sector size larger than 512B, for
  example 4KB. That is, their physical sector size is larger than 512B and they
  do not provide emulation for 512B sector sizes. For such devices both a data
  offset and a data size must be properly aligned. L2ARC should arrange that
  because it uses physical I/O.
  zio_vdev_io_start() performs a necessary transformation if io_size is not
  aligned to vdev_ashift, but that is done only for logical I/O. Something
  similar should be done in L2ARC code.
      * a temporary write buffer should be allocated if the original buffer is
        not going to be compressed and its size is not aligned
      * size of a temporary compression buffer should be ashift aligned
      * for the reads, if a size of a target buffer is not sufficiently large and
        it is not aligned then a temporary read buffer should be allocated

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Saso Kiselkov <saso.kiselkov@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>

MFC after:	3 weeks
2017-06-22 17:10:34 +00:00
avg
8c2cf18ddb MFV r319742: 8056 zfs send size estimate is inaccurate for some zvols
illumos/illumos-gate@0255edcc85
0255edcc85

https://www.illumos.org/issues/8056
  The send size estimate for a zvol can be too low, if the size of the record
  headers (dmu_replay_record_t's) is a significant portion of the size.
  This is typically the case when the data is highly compressible, especially
  with embedded blocks.
  The problem is that dmu_adjust_send_estimate_for_indirects() assumes that
  blocks are the size of the "recordsize" property (128KB).
  However, for zvols, the blocks are the size of the "volblocksize" property
  (8KB). Therefore, we estimate that there will be 16x less record headers than
  there really will be.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Paul Dagnelie <pcd@delphix.com>

MFC after:	3 weeks
2017-06-22 16:58:09 +00:00
avg
b5e511cee1 MFV r318947: 7578 Fix/improve some aspects of ZIL writing.
FreeBSD note: this commit removes small differences between what mav
committed to FreeBSD in r308782 and what ended up committed to illumos
after addressing all review comments.

illumos/illumos-gate@c5ee46810f
c5ee46810f

https://www.illumos.org/issues/7578
  After some ZIL changes 6 years ago zil_slog_limit got partially broken
  due to zl_itx_list_sz not updated when async itx'es upgraded to sync.
  Actually because of other changes about that time zl_itx_list_sz is not
  really required to implement the functionality, so this patch removes
  some unneeded broken code and variables.
  Original idea of zil_slog_limit was to reduce chance of SLOG abuse by
  single heavy logger, that increased latency for other (more latency critical)
  loggers, by pushing heavy log out into the main pool instead of SLOG. Beside
  huge latency increase for heavy writers, this implementation caused double
  write of all data, since the log records were explicitly prepared for SLOG.
  Since we now have I/O scheduler, I've found it can be much more efficient
  to reduce priority of heavy logger SLOG writes from ZIO_PRIORITY_SYNC_WRITE
  to ZIO_PRIORITY_ASYNC_WRITE, while still leave them on SLOG.
  Existing ZIL implementation had problem with space efficiency when it
  has to write large chunks of data into log blocks of limited size. In some
  cases efficiency stopped to almost as low as 50%. In case of ZIL stored on
  spinning rust, that also reduced log write speed in half, since head had to
  uselessly fly over allocated but not written areas. This change improves
  the situation by offloading problematic operations from z*_log_write() to
  zil_lwb_commit(), which knows real situation of log blocks allocation and
  can split large requests into pieces much more efficiently. Also as side
  effect it removes one of two data copy operations done by ZIL code WR_COPIED
  case.
  While there, untangle and unify code of z*_log_write() functions.
  Also zfs_log_write() alike to zvol_log_write() can now handle writes crossing
  block boundary, that may also improve efficiency if ZPL is made to do that.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Steven Hartland <steven.hartland@multiplay.co.uk>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Richard Elling <Richard.Elling@RichardElling.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Alexander Motin <mav@FreeBSD.org>

MFC after:	3 weeks
2017-06-22 16:52:22 +00:00
cem
3f92e78681 sglist.h: Fix sg_refs signedness to match refcount(9)
PR:		220122
Reported by:	Mark Millard <markmi at dsl-only.net>
Sponsored by:	Dell EMC Isilon
2017-06-22 15:52:18 +00:00
emaste
7284d4b0ce retire arm64 kernel module linker workaround
Relocatable linking in aarch64 ld from binutils 2.25.1 does not work.
The linker corrupts the references to the external symbols which are
defined by other object in the linking set and should therefore lose
the GOT entry.

The problem is fixed in later versions of GNU ld and does not exist in
the in-tree lld linker that we now use by default for arm64, so the
workaround can be removed.

Reviewed by:	kib
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11302
2017-06-22 15:09:42 +00:00
emaste
3d6fd4412b Make structure padding explicit in EFI_MEMORY_DESCRIPTOR
The EFI memory descriptor 64-bit aligns PhysicalStart on both 32- and
64-bit platforms.  Make the padding explicit for i386 EFI.

Submitted by:	Siva Mahadevan <smahadevan@freebsdfoundation.org>
MFC after:	3 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D11301
2017-06-22 14:30:09 +00:00
cy
ca30cf090a In poolnodcommand(): TTL (-T) is only valid when adding a node to a
pool (ippool -a) not when removing a node from a pool (ippool -r).
Flag -T as an error in ippool -r.
2017-06-22 12:46:48 +00:00
cy
57e30b47aa poolflush() has no positional arguments. 2017-06-22 06:25:34 +00:00