Commit Graph

1117 Commits

Author SHA1 Message Date
Warner Losh
bb52d711d0 Use the newly minted Makefile.libcompat to implement libsoft libraries
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.
2016-03-12 23:25:05 +00:00
Ed Maste
cb1469fdd6 Bump BOOTSTRAPPING test for libelf after cross-endian fix in r296685 2016-03-12 19:08:13 +00:00
Bryan Drewery
dba75e9fa5 Move Makefile.lib32 to Makefile.libcompat and generalize it.
This is in preparation for LIBSOFT.

This file only supports *1* LIBCOMPAT value currently and must be capitalized.
In Makefile.libcompat given LIBCOMPAT=FOO there can be values set for
LIBFOOCFLAGS, LIBFOOCPUFLAGS, LIBFOOWMAKEENV, LIBFOOWMAKEFLAGS, LIBFOOCPUFLAGS,
and LIBFOODTRACE.  These will have the standard cross-build values appended
onto them.

This could be extended to support multiple libcompat libraries in the future
once there is a need.

Reviewed by:	imp
Sponsored by:	EMC / Isilon Storage Division
Differential Revision:	https://reviews.freebsd.org/D5612
2016-03-11 23:56:33 +00:00
Warner Losh
5202371725 Factor out lib32 generation to its own file. This is prep for a
similar Makefile.libsoft which will do the same for armv6 soft fp API
libraries in prep for pulling the trigger on moving to armv6 hard
float. Once there's two files, I'll work with bdrewery@ to merge the
two files as they are mostly the same. The high rate of churn for
Makefile* makes it quite difficult to make progress out of tree.

Differential Review: https://reviews.freebsd.org/D5566
2016-03-11 03:38:10 +00:00
Bryan Drewery
0481f7852d Don't ever create object directories here with MK_AUTO_OBJ.
Sponsored by:	EMC / Isilon Storage Division
2016-03-08 21:26:25 +00:00
Dimitry Andric
a36b180a7f Merge ^/head r296369 through r296409. 2016-03-05 15:34:54 +00:00
Bryan Drewery
34f4acc8a8 Conditionalize building makewhatis(1) for upgrades that need it.
Remove building of the legacy makewhatis(1) since it was only needed for
6.0 upgrades.

On my 2.2 GHz system libsqlite3 takes 60-100 seconds to build, which due to
its serialized nature can hold up the build waiting on it to finish in
bootstrap-tools.

makewhatis(1) was only required to be a build tool to support upgrades
from 6.0 which was properly removed in r208324 but later reverted due to
installworld using it.  The installworld issue was fixed in r275622
when it was added to ITOOLS.  A BOOTSTRAPPING check was missed when
makewhatis(1) was replaced with mandoc in r283777.

Sponsored by:	EMC / Isilon Storage Division
2016-03-05 05:08:57 +00:00
Bryan Drewery
2bece0ce3d Don't add duplicates from LOCAL_LIB_DIRS.
This causes parallel failures.

Sponsored by:	EMC / Isilon Storage Division
2016-03-04 22:37:16 +00:00
Bryan Drewery
400b94bbeb Fix over/under-linking in contrib/ofed.
Sponsored by:	EMC / Isilon Storage Division
2016-03-04 22:37:09 +00:00
Bryan Drewery
bd6daf52d3 NO_ROOT: Create the METALOG directory on installworld/distributeworld.
When not using NO_ROOT the DESTDIR is implicitly created by the
distrib-dirs call.  However with NO_ROOT it is trying to write
to the METALOG right away before running distrib-dirs which
may fail.

Sponsored by:	EMC / Isilon Storage Division
2016-03-04 22:36:56 +00:00
Ed Maste
6cb64f8497 Add dd as a bootstrap tool, for status= support
In r295757 I added status=none to buildworld's use of dd, but FreeBSD
versions before April 2014 lack support.
2016-03-04 16:03:15 +00:00
Dimitry Andric
82aa34e6fa Merge ^/head r296007 through r296368. 2016-03-03 23:15:46 +00:00
Mariusz Zaborski
c501d73c7e Convert casperd(8) daemon to the libcasper.
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.

Discussed with:		pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by:	drysdale@google.com, bdrewery
Approved by:		pjd (mentor)
Differential Revision:	https://reviews.freebsd.org/D4277
2016-02-25 18:23:40 +00:00
Dimitry Andric
14e9c9161c Merge ^/head r295902 through r296006. 2016-02-24 21:38:51 +00:00
Bryan Drewery
f4d0f67fc9 FAST_DEPEND: Skip 'make depend' for buildworld and kernel since it is auto-ran now.
Sponsored by:	EMC / Isilon Storage Division
2016-02-24 17:20:22 +00:00
Dimitry Andric
9893f787ec Merge ^/head r295601 through r295844. 2016-02-21 13:49:26 +00:00
Bryan Drewery
619a5d7bac Don't descend into test directories for distribution->installconfig.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:14:09 +00:00
Dimitry Andric
90a4726375 Merge ^/head r295544 through r295600. 2016-02-13 16:02:12 +00:00
Ed Maste
c1da297433 Add libpe for elfcopy(1) PE/COFF support
Sponsored by:	The FreeBSD Foundation
2016-02-12 21:12:47 +00:00
Dimitry Andric
4156ce4fed Merge ^/head r295351 through r295543. 2016-02-11 20:07:09 +00:00
Enji Cooper
71b7fa1252 Simplify running the FreeBSD test suite
Replace `make regress` (legacy test make target) and `make test` (incomplete
test make target added with the FreeBSD test suite) with make check as it's
consistent with other open source projects.

`make check` defaults to running tests from `.OBJDIR`, but can be overridden
with the `CHECKDIR` variable.

Add `make checkworld` target to simplify running the FreeBSD test suite from
`TESTSBASE` (i.e. the top-level tests directory), similar to buildworld.

Document `make check` and `make checkworld` in build(7).

Other minor changes:

- Rename intermediate file (`Kyuafile.auto`) to `Kyuafile` to simplify
  `make check`.
- Remove terse warnings attached to `beforetest`/`aftertest`.
- Add kyua binary check to check target in suite.test.mk; error out if it's
  not found

The MFC is [partly] contingent on other build related changes being MFCed.

Differential Revision: https://reviews.freebsd.org/D4406
MFC after: 2 months
X-MFC to: stable/10
Relnotes: yes
Reviewed by: bdrewery, Evan Cramer <eccramer@gmail.com>
Sponsored by: EMC / Isilon Storage Division
2016-02-07 18:40:04 +00:00
Dimitry Andric
a49d8b6ecc Merge ^/head r294961 through r295350. 2016-02-06 14:07:17 +00:00
Bryan Drewery
db9ba9ee1b Deduplicate distrib-dirs logic from r289086 in distribution.
This does it correctly this time, rather than the incorrect version in
r295167.

Sponsored by:	EMC / Isilon Storage Division
2016-02-03 20:24:21 +00:00
Conrad Meyer
4be4b11f30 Revert r295167 at bdrewery's request
$ svn merge -c -295167 .

JHB reports Navdeep reports that it breaks distribution and etcupdate.

Approved by:	bdrewery
2016-02-03 01:40:07 +00:00
Bryan Drewery
fb872bbc68 Deduplicate distrib-dirs logic from r289086 in distribution.
Sponsored by:	EMC / Isilon Storage Division
2016-02-02 21:11:23 +00:00
Bryan Drewery
7cd7d12629 Adjust install .WAITs for lib/ and etc/ to allow parallelization more.
Only 'installworld' needs to be protected and only when not using
-DNO_ROOT, which implies not installing to / and not needing the
lib dependency protections.

Sponsored by:	EMC / Isilon Storage Division
2016-02-02 20:50:09 +00:00
Dimitry Andric
ea2c42d8d3 Merge ^/head r293686 through r293849. 2016-01-13 20:16:53 +00:00
Bryan Drewery
57ce33acfe Correct a comment.
Submitted by:	jhb
2016-01-11 20:27:05 +00:00
Dimitry Andric
89d3f0ea4e Merge ^/head r293430 through r293685. 2016-01-11 19:36:44 +00:00
Bryan Drewery
1f17dba482 Chase r292622: Update path to ioctl.c for incremental build hack.
Sponsored by:	EMC / Isilon Storage Division
2016-01-08 22:59:49 +00:00
Bryan Drewery
8dc7b23a02 Fix upgrading from OSVERSION 1000002-1000032 after r288829.
r288829 states that lex requires the latest m4, but was not always building it.
Move lex to the same logic as m4 since they are closely tied now.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
Reported by:	Slawa Olhovchenkov <slw@zxy.spb.ru>
2016-01-08 21:07:34 +00:00
Dimitry Andric
b229c1a0b0 Merge ^/head r293280 through r293429. 2016-01-08 17:42:14 +00:00
Gleb Smirnoff
b46e917554 Provide knob NO_INSTALLEXTRAKERNELS. If defined, extra kernels in KERNCONF
won't be installed, only the first one would.
2016-01-06 21:58:45 +00:00
Dimitry Andric
5dc6cc42f4 Merge ^/head r293175 through r293279. 2016-01-06 21:31:07 +00:00
Glen Barber
8834318685 Merge ^/projects/release-install-debug:
- Rework MANIFEST generation and parsing via bsdinstall(8).
- Allow selecting debugging distribution sets during install.
- Rework bsdinstall(8) to fetch remote debug distribution sets
  when they are not available on the local install medium.
- Allow selecting additional non-GENERIC kernels during install.
  At present, GENERIC is still required, and installed by default.

Tested with:	head@r293203
Sponsored by:	The FreeBSD Foundation
2016-01-05 21:05:17 +00:00
Dimitry Andric
34cdd77646 First part of updating llvm/clang build glue: getting llvm-tblgen,
clang-tblgen and clang itself built.
2015-12-31 14:39:45 +00:00
Sean Bruno
7144d5cbc5 Fixup native-xtools target for poudriere cross build jails after svn
r291955 by excluding all debug files.

Reported by:	swills

Reviewed by:	emaste
2015-12-23 21:55:54 +00:00
Enji Cooper
4a2a740e17 Always expose LOCALBASE, not just when CROSS_TOOLCHAIN is defined
Instead of using which(1) to look for doxygen, look for it in <LOCALBASE>/bin .
$PATH gets mangled by make buildenv, etc so it's better to just be explicit
about the path if someone uses that for instance.

Differential Revision: https://reviews.freebsd.org/D4406 (part of a larger diff)
Reviewed by: emaste, Evan Cramer <eccramer@gmail.com>
Sponsored by: EMC / Isilon Storage Division
2015-12-20 05:52:03 +00:00
Bryan Drewery
37ded2a72c Follow-up r290423: Don't use CSH for buildenv shell.
It does not properly import PATH; the PATH is reset by included profile
files on startup which breaks the biggest feature of buildenv (using
sysrooted cc from WORLDTMP)

Spotted by:	smh, kib
Sponsored by:	EMC / Isilon Storage Division
2015-12-14 23:25:31 +00:00
Andrew Turner
c492eb76dd Split KNOWN_ARCHES so we have one per line to simplify merging future
architectures.
2015-12-14 09:52:09 +00:00
Ruslan Bukin
1fdcc5e5c0 Start support for the RISC-V 64-bit architecture developed by UC Berkeley.
RISC-V is a new ISA designed to support computer research and education, and
is now become a standard open architecture for industry implementations.

This is a minimal set of changes required to run 'make kernel-toolchain'
using external (GNU) toolchain.

The FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv.

Reviewed by:	andrew, bdrewery, emaste, imp
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D4445
2015-12-11 22:55:23 +00:00
Bryan Drewery
9229b1797a Fix some makeman issues.
- Don't bother looking up REVISION/BRANCH/etc from release/, or the
  CPUTYPE check, as these are not used for makeman and wastes time.  The also
  invokes auto.obj.mk after I reverted auto.obj.mk ignoring -V in r291312.
- Don't modify CC or PATH when WITH_CCACHE_BUILD or WITH_META_MODE is enabled
  as it leads to bsd.compiler.mk errors.

Sponsored by:	EMC / Isilon Storage Division
2015-12-08 17:09:17 +00:00
Bryan Drewery
1c2a618bbf Fix spelling of internal hack.
Reported by:	ngie
2015-12-07 18:45:55 +00:00
Bryan Drewery
03173d2ff0 bsd.subdir.mk: Only recurse on called targets, rather than dependencies.
This is to fix 'make all' causing it to recurse on both 'all' and 'buildconfig'
due to 'buildconfig' being in ALL_SUBDIR_TARGETS and being a dependency of
'all'.

This now adds all of the '*includes', '*files' targets as subdir targets,
allowing them to recurse.

This also removes the need for some 'realinstall' hacks in bsd.subdir.mk since
it no longer recurses; only 'install' will recurse and call the proper
'beforeinstall', 'realinstall', and 'afterinstall' in each sub-directory.

This fixes 'make includes' and 'make files' to not be a rerolled ${MAKE}
sub-shell but to rather just recurse on 'inclues' and 'files'.  This avoids
various issues such as the one fixed in r289462.  As such revert Makefile.inc1
back to using 'includes' which avoids an extra tree walk and parallelizes
the includes phases better.

Makefile.inc1 includes a guard so that 'make all' will not use SUBDIR_PARALLEL,
added in r289438.  This is so users do not get a probably broken build if they
run 'make all' from the top-level.  Before the change in this commit, the
workaround for 'make everything' was 'par-all' which would depend on 'all' and
cause a proper parallel recursion.  Now that will not work so a new
_PARALLEL_SUBUDIR_OK is used to allow it.

This is still part of an effort to combine bsd.(files|incs|confs).mk and move
some of its logic out of bsd.subdir.mk, as attempted in r289282 and reverted in
r289331.  This commit fixes the problems found there which was mostly double
recursing during 'includes' which would recurse on itself and 'buildincludes'
and 'installincludes', all in parallel.  The logic is still in bsd.subdir.mk
for now.

I've been cautious about this commit but have experienced no breakage on the
tree except for the 'par-all' case which was already a hack.  If something foo
is depending on something bar that should recurse, it is very likely that the
foo target is being recursed on already meaning that bar will still effectively
recurse once sub-directories call foo.

Discussed on:	arch@
MFC after:	never
Sponsored by:	EMC / Isilon Storage Division
2015-12-02 01:50:22 +00:00
Bryan Drewery
6fa9702035 Add NO_INSTALLKERNEL to undo the assumption that the first KERNCONF will be
installed as "kernel".  This is relevant for packaging of the kernel when
not wanting a default "kernel.txz".

Submitted by:	Russell Cattelan <cattelan@thebarn.com>
MFC after:	2 weeks
Obtained from:	OneFS
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 20:24:58 +00:00
Bryan Drewery
0a63717cfa Allow storing package(world|kernel) tarballs into a different location at PACKAGEDIR.
Submitted by:	Russell Cattelan <cattelan@thebarn.com>
Discussed with:	gjb
MFC after:	2 weeks
Obtained from:	OneFS
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 19:58:08 +00:00
Bryan Drewery
b67490a3b0 Fix errors being ignored in many phases of the build since the bmake integration.
Say it with me, "I will not chain commands with && in Makefiles"

This was originally fixed and explained quite well by bde@ in r36074.  The
initial bmake integration caused 'set -e' to stop being used which lead to
r252419.  Later 'set -e' expectations were fixed with bmake in r254980.

Because of the && here, errors would be ignored when building in parallel and
a dependency failed.  Such as bootstrap-tools since it builds everything in
parallel.  If any tool failed in obj/depend/all, it would just ignore the error
and continue to build.  This later would result in cascaded errors that only
confused the real issue.  This could also cause commands after the failed
command to still execute, leading to more confusion.

This should be fine if the command is in a sub-shell such as: (cmd1 && cmd2)

This reverts r252419.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2015-12-01 19:00:43 +00:00
John Baldwin
7f911abe54 Add support to libkvm for reading vmcores from other architectures.
- Add a kvaddr_type to represent kernel virtual addresses instead of
  unsigned long.
- Add a struct kvm_nlist which is a stripped down version of struct nlist
  that uses kvaddr_t for n_value.
- Add a kvm_native() routine that returns true if an open kvm descriptor
  is for a native kernel and memory image.
- Add a kvm_open2() function similar to kvm_openfiles().  It drops the
  unused 'swapfile' argument and adds a new function pointer argument for
  a symbol resolving function.  Native kernels still use _fdnlist() from
  libc to resolve symbols if a resolver function is not supplied, but cross
  kernels require a resolver.
- Add a kvm_nlist2() function similar to kvm_nlist() except that it uses
  struct kvm_nlist instead of struct nlist.
- Add a kvm_read2() function similar to kvm_read() except that it uses
  kvaddr_t instead of unsigned long for the kernel virtual address.
- Add a new kvm_arch switch of routines needed by a vmcore backend.
  Each backend is responsible for implementing kvm_read2() for a given
  vmcore format.
- Use libelf to read headers from ELF kernels and cores (except for
  powerpc cores).
- Add internal helper routines for the common page offset hash table used
  by the minidump backends.
- Port all of the existing kvm backends to implement a kvm_arch switch and
  to be cross-friendly by using private constants instead of ones that
  vary by platform (e.g. PAGE_SIZE).  Static assertions are present when
  a given backend is compiled natively to ensure the private constants
  match the real ones.
- Enable all of the existing vmcore backends on all platforms.  This means
  that libkvm on any platform should be able to perform KVA translation
  and read data from a vmcore of any platform.

Tested on:	amd64, i386, sparc64 (marius)
Differential Revision:	https://reviews.freebsd.org/D3341
2015-11-27 18:58:26 +00:00
Bryan Drewery
20f680588a Remove unneeded libmd from bootstrap-tools (reverting r246784).
The bootstrap-tools are supposed to be host tools, which in most cases, use
host headers and libraries.  As such, directly including the src tree's headers
for libmd here causes the need to link libmd in since it will be built with
the new symbols (which /usr/lib/libmd.so) won't have unless it is new enough.
During the target build in buildworld the target headers are staged into
WORLDTMP and used via --sysroot, allowing the target xinstall to be built with
the new/target libmd.

The .PATH here was also not doing anything since xinstall does not use libmd
source files.

Sponsored by:	EMC / Isilon Storage Division
MFC after:	2 weeks
2015-11-18 21:37:46 +00:00
Bryan Drewery
f2307a2a25 Avoid setting schg in the objtree for lib32 build.
Reported by:	kib
Sponsored by:	EMC / Isilon Storage Division
MFC after:	1 week
2015-11-14 01:45:37 +00:00