r364166 converted echo -n `/bin/pwd` to a raw pwd invocation, leaving a
trailing newline at the end of path. This caused a later stat() of it to
erroneously fail and the fallback to MK_AUTO_OBJ=no logic proceeded as
unexpected.
Harry Schmalzbauer bissected the resulting build failure he experienced
(stable/12 host, -HEAD build) down to r365887. This change is mostly
unrelated, except it switches the build to bootstrapped crunchgen - clue!
I then bissected recent crunchgen changes going back a bit since we wouldn't
observe the failure immediately with -CURRENT in most configurations, which
landed me on r364166. After many intense head-scratching minutes and printf
debugging, I realized that the newline was the difference. This is where our
tale ends.
Reported by: Harry Schmalzbauer, O. Hartmann, Mike Tancsa, kevans
MFC after: 3 days
This fixes a compile issue under gcc6 which complains about
legacy style C function declarations.
Differential Revision: https://reviews.freebsd.org/D26504
This was revealed by the rescue build with a patch I'm working on to default
WARNS=6 everywhere. The issues resolved were:
- Missing prototype for _crunched_${ident}_stub in the *_stub.c generated
bits
- Missing prototype for crunched_main
- Incomplete prototype for _crunched_${ident}_stub in the generated parts of
crunched_main
- Literal strings in the stub table must drop const qualifier, unless we
const'ify name
- f field in struct stub didn't have a proper prototype
Most of these issues are minor formalities and easily addressed.
I note that if my patch to eventually raise WARNS for the rescue build
lands, we'll need to bump the __FreeBSD_version requirement for
bootstrapping crunchgen and wipe out the rescue .OBJDIR if it's stale, which
we should be able to detect pretty easily from a couple of the issues that
have been fixed here.
Reviewed by: arichardson
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26363
This should fix cases like su setting argv[0] to _su for /bin/sh.
Previously cheribsdbox (a crunched tool we use in CheriBSD to reduce the
size of our minimal disk images to allow loading them onto FPGAs without
waiting forever for the transfer) would complain about _su not being
compiled in, but now that we also look at AT_EXECPATH it correctly
invokes the sh tool.
Note: we use use AT_EXECPATH instead of the KERN_PROC_PATHNAME sysctl to get
the crunchgen binary name since it seems like KERN_PROC_PATHNAME just
returns the last cached path for a given hardlink.
When using `su`, instead of invoking /bin/csh this would invoke the last
used hardlink to cheribsdbox. This caused weird test failures when running
tests due to `id` being executed instead of `echo`:
$ id # id is a hardlink to /bin/cheribsdbox
$ su postgres -c 'echo 1' # su is also a hardlink
uid=1001(postgres) gid=1001(postgres) groups=1001(postgres)
Obtained from: CheriBSD
Reviewed By: emaste, brooks
Differential Revision: https://reviews.freebsd.org/D25998
The -P flag is required by POSIX so we don't have to care whether pwd is
a shell builtin or not. This also allows removing pwd from the list of
bootstrap tools since all shells we care about for building have a
builtin pwd command. This effectively reverts r364190.
Suggested By: rgrimes, jrtc27
In r364166 I changed /bin/pwd to pwd, but pwd can be shell builtin that
may not correctly return a real path. To ensure that all symlinks are
resolved use `env pwd -P` instead (the -P flag is part of POSIX so
should be supported everywhere).
Reported By: rgrimes
Suggested By: jrtc27
On Glibc systems mkstemp can only be used once with the same template
string since it will be modified in-place and no longer contain any 'X' chars.
It is fine to reuse the same file here but we need to be explicit and use
open() instead of mkstemp() on the second use.
While touching this file also avoid a hardcoded /bin/pwd since that may not
work when building on non-FreeBSD systems.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D25990
rcmds removed in r324351.
Historical references in the README are maintained. There's a paragraph
describing a "980K crunched 'fixit'" that references rsh and rlogin.
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
No functional change intended.
Failure modes of the modern (that is, produced in the last 25 years)
hard drives and SSDs made the utility outdated. Since the kernel
interface to support it was removed in r324853, cut the userspace
remnants as well.
Discussed with: bde (who does not like the removal)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
in favor of just rendering the manpage instead of relying on pre-formatted
catpages. Note, this does not impede the ability to use existing catpages,
it just removes the utility to generate them.
Reviewed by: imp, allanjude
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D12317
Some tools produce objects with a combined strtab and shstrtab.
These objects are not supported by crunchide since it rewrites the
symtab and strtab to "hide" symbols. This invalidates section header
offsets into a combined strtab/shstrtab.
In the future we could support these objects (by ensuring that we retain
unmodified section name strings in the output .strtab, and then rewriting
each section header's sh_name).
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
- DIRDEPS_BUILD: Fix crunchgen builds losing their library dependencies
on a nop-rebuild.
- META_MODE: Fix not rebuilding various crunch.mk targets if their .meta
files warrant a rebuild. They were lacking .meta files previously.
This adds .NOMETA to the crunch objects being used since they are
already built. Bmake was forcing a rebuild on them since their
.meta files were not in the expected place; there is no reason to
rebuild them.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Passing MAKEOBJDIRPREFIX to the main prog build (rescue) would confuse
WITH_AUTO_OBJ and cause it to create a recursed object directory that
then broke the actual prog build. This is normally not a problem since
we do not call 'make -f prog.mk obj' before building anything in it.
Crunchgen(1) also assumes that if -o is not passed then if an object
directory does not already exist then it should build in the source
directories. The normal buildworld process will have already ran 'make
obj' in each of the component directories so this is not a problem.
With WITH_AUTO_OBJ though this is not the case. So we must tell
crunchgen(1) that MK_AUTO_OBJ=yes will create the directory and to not
require it be present before generating its Makefile.
Sponsored by: EMC / Isilon Storage Division
These are no longer needed after the recent 'beforebuild: depend' changes
and hooking DIRDEPS_BUILD into a subset of FAST_DEPEND which supports
skipping 'make depend'.
Sponsored by: EMC / Isilon Storage Division
This was causing the following error:
rescue
sh: rescue: not found
*** [rescue] Error code 127
make[1]: stopped in /usr/obj/usr/src/rescue/rescue
Sponsored by: EMC / Isilon Storage Division
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
Most of the EM_* constants are available in all supported host branches,
but EM_AARCH64 was added relatively recently. Add it back to fix
building HEAD on 10.x.
Noticed by: adrian, jmallett
This avoids the need to build a target-specific crunchide for cross-
uilds.
Differential Revision: https://reviews.freebsd.org/D2314
Sponsored by: The FreeBSD Foundation
For 64-bit binaries the Elf_Ehdr e_shoff is at offset 40, not 44.
Instead of using an incorrect hardcoded offset, let the compiler
figure it out for us with offsetof().
Differential Revision: https://reviews.freebsd.org/D1543
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
1. 50+% of NO_PIE use is fixed by adding -fPIC to INTERNALLIB and other
build-only utility libraries.
2. Another 40% is fixed by generating _pic.a variants of various libraries.
3. Some of the NO_PIE use is a bit absurd as it is disabling PIE (and ASLR)
where it never would work anyhow, such as csu or loader. This suggests
there may be better ways of adding support to the tree. Many of these
cases can be fixed such that -fPIE will work but there is really no
reason to have it in those cases.
4. Some of the uses are working around hacks done to some Makefiles that are
really building libraries but have been using bsd.prog.mk because the code
is cleaner. Had they been using bsd.lib.mk then NO_PIE would not have
been needed.
We likely do want to enable PIE by default (opt-out) for non-tree consumers
(such as ports). For in-tree though we probably want to only enable PIE
(opt-in) for common attack targets such as remote service daemons and setuid
utilities. This is also a great performance compromise since ASLR is expected
to reduce performance. As such it does not make sense to enable it in all
utilities such as ls(1) that have little benefit to having it enabled.
Reported by: kib
This includes:
o All directories named *ia64*
o All files named *ia64*
o All ia64-specific code guarded by __ia64__
o All ia64-specific makefile logic
o Mention of ia64 in comments and documentation
This excludes:
o Everything under contrib/
o Everything under crypto/
o sys/xen/interface
o sys/sys/elf_common.h
Discussed at: BSDcan
This is currently an opt-in build flag. Once ASLR support is ready and stable
it should changed to opt-out and be enabled by default along with ASLR.
Each application Makefile uses opt-out to ensure that ASLR will be enabled by
default in new directories when the system is compiled with PIE/ASLR. [2]
Mark known build failures as NO_PIE for now.
The only known runtime failure was rtld.
[1] http://www.bsdcan.org/2014/schedule/events/452.en.html
Submitted by: Shawn Webb <lattera@gmail.com>
Discussed between: des@ and Shawn Webb [2]
