Commit Graph

86 Commits

Author SHA1 Message Date
bdrewery
7f1e5a22b7 AUTO_OBJ: Don't create nested OBJDIRS with print-dir or make -n.
Sponsored by:	Dell EMC
2017-12-06 21:00:41 +00:00
bdrewery
e10157c915 Handle some .OBJDIR == .CURDIR cases.
- If OBJROOT is SRCTOP then don't add on TARGET.TARGET_ARCH.  This
  only happens at the top-level, and for sub-directories when the
  user is clever with MAKEOBJDIRPREFIX=/.
- Don't bother checking 'test -w' on .CURDIR.
- Properly set OBJTOP/OBJROOT to SRCTOP in various needed cases.
- Check if the OBJDIR is writable even for *clean* targets since it
  determines which .OBJDIR the user gets;  If they cannot write to an
  existing eligible .OBJDIR then it needs to clean in .CURDIR instead.
- Add guard to cleanworld/cleanuniverse from removing SRCTOP.
- Ensure OBJTOP is proper for .OBJDIR=.CURDIR which fixes finding
  libraries since src.libnames.mk is based on OBJTOP.
- Avoid some chdir(2) for modifying .OBJDIR

Sponsored by:	Dell EMC Isilon
2017-11-10 02:09:33 +00:00
bdrewery
039afc98f9 Mark targets .PHONY.
This avoids the obvious of not running the target when expected, but
also avoids META_MODE from showing 'Building'.  This is mostly only
a problem when directly including bsd.obj.mk as many of these targets
were already .PHONY via bsd.sys.mk.

Sponsored by:	Dell EMC Isilon
2017-11-09 22:08:07 +00:00
bdrewery
9963a47393 Follow-up r325405: Fix assertion still for kernel module builds.
Apparently they have SRCTOP set, but won't have RELDIR set since they are
not in-tree.

Pointyhat to:	bdrewery
Reported by:	O. Hartmann <ohartmann@walstatt.org>, dhw
Tested by:	dhw
Sponsored by:	Dell EMC Isilon
2017-11-05 16:28:05 +00:00
bdrewery
602b873130 AUTO_OBJ: Allow 'make cleandir obj all' combinations.
The cleanobj target will not really remove the OBJDIR in this case,
it will only remove the OBJDIR if only clean targets are ran.

Sponsored by:	Dell EMC Isilon
2017-11-05 00:12:12 +00:00
bdrewery
bc0048e6ef AUTO_OBJ: Set .OBJDIR=.CURDIR earlier for NO_OBJ Makefiles.
auto.obj.mk is loaded before the Makefile which may have NO_OBJ set inside of
it.  In that case we can't avoid creating the OBJDIR but we do need to avoid
using it.  Fixing .OBJDIR in bsd.init.mk at least fixes all of the build .mk
files to have a proper .OBJDIR.  It does not fix the Makefile itself but
usually if they have NO_OBJ set they are not inspecting ${.OBJDIR} anyhow.

It is ideal to only have this in bsd.init.mk and to remove it from bsd.obj.mk,
but then bsd.obj.mk would need to include bsd.init.mk.  Doing something like
that would require more testing.  It has been proposed that bsd.obj.mk not be
included directly but it has been possible for too long to do so to change it
at this point.

Note too that it may make sense to fix .OBJDIR even when AUTO_OBJ is not
enabled but the historical behavior has always been that NO_OBJ just avoids
running 'make obj', so .OBJDIR should already be .CURDIR.

Also while NO_OBJ seems like it should be removed at this point, it is not
always possible to fix Makefiles to properly use an .OBJDIR.  The cost of
keeping NO_OBJ support is minimal.

Sponsored by:	Dell EMC Isilon
2017-11-05 00:11:58 +00:00
bdrewery
872d34708e Follow-up r325405: Only enable assertion for in-tree src builds.
Pointyhat to:	bdrewery
Reported by:	O. Hartmann <ohartmann@walstatt.org>
Sponsored by:	Dell EMC Isilon
2017-11-04 21:59:54 +00:00
bdrewery
c3e293f179 Add an assertion to avoid the 'rm -rf /etc' case from cleanobj fixed in r325404.
Sponsored by:	Dell EMC Isilon
2017-11-04 21:05:03 +00:00
bdrewery
8800d532d3 Add option UNIFIED_OBJDIR, on by default, which moves the default build OBJDIR.
This changes the build OBJDIR from the older style of /usr/obj/<srcdir> for
native builds, and /usr/obj/<target>.<target_arch>/<srcdir> for cross builds to
a new simpler format of /usr/obj/<srcdir>/<target>.<target_arch>.  This
new format is used regardless of cross or native build.  It allows
easier management of multiple source tree object directories.

The UNIFIED_OBJDIR option will be removed and its feature made permanent
for the 12.0 release.

Relnotes:	yes (don't note UNIFIED_OBJDIR option since it will be removed)
Prior work:	D3711 D874
Reviewed by:	gjb, sjg
Discussed at:	https://lists.freebsd.org/pipermail/freebsd-arch/2016-May/017805.html
Discussed with:	emaste
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12840
2017-11-01 21:22:05 +00:00
bdrewery
9628714bcb cleandir: Fix ESTALE errors from parallel removals.
This fixes 'make cleandir' to use the same ordering as 'make cleanobj'.
Meaning that SUBDIR will be recursed before the current directory is
handled.  This avoids an 'rm -rf /usr/obj/usr/src/lib/libc' while
a child 'rm -rf /usr/obj/usr/src/lib/libc/tests' is being ran next,
or even removing the current directory and then recursing into a child
and using the 'missing OBJDIR' logic to remove files rather than the
directory.

The most ideal ordering here would be for 'cleanobj' and 'cleandir' to
simply remove the .OBJDIR and then not recurse at all.  This is only
safe if it is guaranteed that all children directories have no orphaned
files in their source checkout and are only using obj directories.  This
is usually safe from the top-level build targets and when using
WITH_AUTO_OBJ.  Improving the build for those cases is coming.

Reported by:	cperciva, scottl
X-MFC-With:	r321427
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-25 00:12:48 +00:00
bdrewery
635b1e3eb5 PROGS: Fix ESTALE errors on NFS while cleaning in directories with PROGS.
- Only recurse on cleanobj/cleandir if there is no .OBJDIR being used.
  If we don't recurse then bsd.obj.mk will just rm -rf the OBJDIR dir.
- When recursing on cleanobj/cleandir don't remove dependfiles/dirs
  redundantly from the child and main processes.  Meaning '.depend', and
  'tags', and '.depend.*' will now only be removed from the main
  process.
- Stop recursing on 'cleandepend' since the main process can handle
  removing all files via the default glob patterns in CLEANDEPENDFILES.
- This reverts r288201, by readding recursion on 'cleanobj', due to
  r291635 changing how bsd.subdir.mk handles recursion.

This is primarily targeting ESTALE NFS errors from rm(1) during a
buildworld but is also a performance optimization as both issues fixed
were redundant anyway.

Reported by:	cperciva, scottl
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 17:54:03 +00:00
bdrewery
0355687f0f cleanobj: Unhide removal of directory.
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-07-24 17:53:45 +00:00
bdrewery
4f4f919cf3 objwarn should be .PHONY.
Otherwise in META_MODE it may create an objwarn.meta if only bsd.obj.mk
is included;  bsd.sys.mk already had .PHONY: objwarn.

MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2017-06-21 17:11:49 +00:00
bdrewery
7ff669ea6a AUTO_OBJ: Support SRCS/DPSRCS that contain '/' and require a nested OBJDIR.
This is to be used by the new clang3.9 build and extends functionality
added to 'make obj' in r279980.

MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2016-08-30 22:48:05 +00:00
bdrewery
d5d515e9ff Allow parallelizing clean/cleandepend/depend with all.
This now makes 'make -jX clean all install' safe.

MFC after:	3 days
Sponsored by:	EMC / Isilon Storage Division
2016-08-29 18:31:34 +00:00
dim
cb0b718c02 In addition to creating subdirectories under .OBJDIR for SRCS with
relative paths, also create them for DPSRCS.  This is needed for builds
that generate files during the depend stage, which cannot be compiled by
themselves, since those have to be put in DPSRCS.
2016-08-23 19:31:43 +00:00
bdrewery
6943e659c0 Consolidate common beforebuild logic.
Sponsored by:	EMC / Isilon Storage Division
2016-02-16 02:09:48 +00:00
bdrewery
77a4b98b93 Standardize on OBJTOP in and outside of META MODE.
Sponsored by:	EMC / Isilon Storage Division
2015-11-26 01:47:56 +00:00
sjg
95ce3d5283 With MK_AUTO_OBJ=yes objdir is created during sys.mk
If a makefile sets NO_OBJ, we should honor that by setting
.OBJDIR back to .CURDIR
2015-06-11 13:53:15 +00:00
sjg
9eb4953043 combine two comments 2015-06-10 19:31:10 +00:00
sjg
79080dd82e Explain in a nutshell why we destroy 2015-06-10 19:26:49 +00:00
sjg
55b0578965 If MK_AUTO_OBJ==yes, it is done before bsd.obj.mk included 2015-06-10 18:09:05 +00:00
sjg
60ff4eb0df Merge from head 2015-05-26 21:52:57 +00:00
dim
d202cb2c64 Allow relative pathnames in SRCS, so as to enable building software
which includes more than one file with the same name, in different
directories.

For example, setting:

SRCS+=	foo/foo.c bar/foo.c baz/foo.c

will now create separate objdirs 'foo', 'bar' and 'baz' for each of the
sources in the list, and use those objdirs for the corresponding object
files.

Reviewed by:	brooks, imp
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D1984
2015-03-14 12:29:44 +00:00
sjg
b137080f19 Merge from head@274682 2014-11-19 01:07:58 +00:00
imp
b64e45fa2d Ignore errors from rm -rf to support high -j builds. This is, at best,
a kludge. However, it also effectively works around the issues for
high -j builds on systems that do not have the rm fixes.

A better fix would be to rmdir here, and fix the places where we're
sloppy and not list all the files we create in CLEANFILES, should
anybody have the time to chase them all to ground.
2014-11-06 17:19:41 +00:00
sjg
d7cd1d425c Merge head from 7/28 2014-08-19 06:50:54 +00:00
sjg
4c49a9e5c9 bsd.lib.mk does not add OBJS etc to CLEANFILES so does not automatically
get them flagged as .NOPATH.  This hurts people who don't use obj dirs.
Since its clean target seprate rm's for things, use NOPATH_FILES as list
to collect things that need .NOPATH.

bsd.obj.mk will add CLEANFILES to NOPATH_FILES and do the deed if needed.

Reviewed by: sbruno
2014-07-22 00:42:55 +00:00
sjg
8202a43f30 Don't make wildcards .PHONY
Add pseudo machines host and common to machine list for destroy
2014-05-10 00:24:32 +00:00
sjg
d200229887 If doing destroy-all and at top-level whack stage tree too. 2014-05-05 18:17:26 +00:00
sjg
1911be667b Add 'destroy' target to efficiently rm -rf objtree
Reviewed by: obrien
2014-05-02 01:39:30 +00:00
marcel
52e7fd7c54 Add support for bmake. This includes:
1.  Don't do upgrade_checks when using bmake. As long as we have WITH_BMAKE,
    there's a bootstrap complication in ths respect. Avoid it. Make the
    necessary changes to have upgrade_checks work wth bmake anyway.
2.  Remove the use of -E. It's not needed in our build because we use ?= for
    the respective variables, which means that we'll take the environment
    value (if any) anyway.
3.  Properly declare phony targets as phony as bmake is a lot smarter (and
    thus agressive) about build avoidance.
4.  Make sure CLEANFILES is complete and use it on .NOPATH. bmake is a lot
    smarter about build avoidance and should not find files we generate in
    the source tree. We should not have files in the repository we want to
    generate, but this is an easier way to cross this hurdle.
5.  Have behavior under bmake the same as it is under make with respect to
    halting when sub-commands fail. Add "set -e" to compound commands so
    that bmake is informed when sub-commands fail.
6.  Make sure crunchgen uses the same make as the rest of the build. This
    is important when the make utility isn't called make (but bmake for
    example).
7.  While here, add support for using MAKEOBJDIR to set the object tree
    location. It's the second alternative bmake looks for when determining
    the actual object directory (= .OBJDIR).

Submitted by:	Simon Gerraty <sjg@juniper.net>
Submitted by:	John Van Horne <jvanhorne@juniper.net>
2012-10-06 20:01:05 +00:00
marcel
9dd41e3647 Sync FreeBSD's bmake branch with Juniper's internal bmake branch.
Requested by: Simon Gerraty <sjg@juniper.net>
2012-08-22 19:25:57 +00:00
harti
f2968c80c9 Make sure the only thing that follows .endif or .else is a comment. 2005-04-11 07:13:29 +00:00
krion
09885bc66a Remove trailing spaces. 2005-01-06 11:12:43 +00:00
ru
cec60429bb Start the dreaded NOFOO -> NO_FOO conversion.
OK'ed by:	core
2004-12-21 08:47:35 +00:00
ru
0807140c6b Mark the "obj" target with the .PHONY attribute. 2004-04-13 12:36:00 +00:00
green
7555fe73fd Fix a very corner case when you want to make cleandir SUBDIRs which
are built using a ${MAKE} that's not just "make".

Test by:	make universe (followed by cleandirs)
2004-01-02 05:05:48 +00:00
kris
ea92fd0fc3 test -h is deprecated; use -L instead.
Submitted by:	april <april@oublinet.net>
PR:		misc/38724 (part of)
MFC after:	3 days
2002-08-31 02:22:33 +00:00
ru
3d25ed729f Moved the `distribute' target from bsd.obj.mk to bsd.subdir.mk,
to make it call `install' in the bsd.subdir.mk-driven makefiles
too.  (share/examples/Makefile,v 1.29 changed the bsd.prog.mk
to bsd.subdir.mk and many stuff was lost during "make release".
I then merged this change in rev. 1.28.2.2 to work around the
namespace pollution (FILES) in this makefile.)

There was an added complexity here.  Both the `distribute' and
`install' targets are recursive (they propagate to SUBDIRs).
So `distribute' first calls `install' in the ${.CURDIR}, then
calls `distribute' in each SUBDIR, etc.  The problem is that
`install' (being also recursive) causes the stuff from SUBDIR
to be installed twice, first time thru `install' in ${.CURDIR}
triggered by `distribute', second time by `distribute' run in
the SUBDIR.  This problem is not new, but it became apparent
only after I moved the `distribute' target from bsd.obj.mk to
bsd.subdir.mk.  My first attempt testing the fix failed due to
this, because the whole world was distributed twice, causing
all the imaginable mess (kerberos5 stuff was installed into both
"base" and "krb5" dists, there was /sbin/init.bak, etc.)
I say the problem is not new because bsd.prog.mk and bsd.lib.mk
makefiles with SUBDIR (even without this fix) had this problem
for years.  Try e.g. running ``make distribute DISTDIR=/foo''
from usr.bin/bzip2 or from lib/libcom_err (without the fix) and
watch the output.

So the solution was to make `install' behave non-recursive when
executed by `distribute'.  My first attempt in passing SUBDIR=
to the `install' in the `distribute' body failed because of the
way how src/Makefile and src/Makefile.inc1 communicate with each
other.  SUBDIR='s assignment precedence on the "make install
SUBDIR=" command line is lowered after src/Makefile wrapper calls
"make ... -f ${.CURDIR}/Makefile.inc1 install" because SUBDIR=
is moved into environment, and Makefile.inc1's assignments now
take higher precedence.  This may be fixed someday when we merge
Makefile with Makefile.inc1.  For now, this is implemented as a
NO_SUBDIR knob.

Spotted by:	Dmitry Pryanishnikov <dmitry@atlantis.dp.ua>
Prodded by:	des
MFC after:	3 days
2002-07-12 15:09:35 +00:00
ru
881d4d841b Moved checkdpadd target to where it logically belongs. 2002-07-03 12:44:06 +00:00
ru
cbabda3807 Fixed typos. 2002-07-01 07:25:02 +00:00
ru
3c39d32871 Quiet ``make objlink'' when NOOBJ is defined.
PR:		bin/21142
Submitted by:	Craig Leres <leres@ee.lbl.gov>
2002-06-21 07:30:51 +00:00
ru
040f231e67 Abuse bsd.obj.mk for defining default distribute target. 2002-04-24 13:29:34 +00:00
ru
1a085bedc0 Merge bsd.obj.mk's version of the _SUBDIR target with bsd.subdir.mk.
Ensure all standard targets honor SUBDIR.  Now `make obj' descends into
SUBDIRs even if NOOBJ is set (some descendants may still need an object
directory, but we do not have such precedents).  Now `make install' in
non-bsd.subdir.mk makefiles runs `afterinstall' target _after_ `install'
in SUBDIRs, like we do in bsd.subdir.mk.  Nothing depended on the wrong
order anyway.

Fixed `distribute' targets (except for the bsd.subdir.mk version) so that
they do not depend on _SUBDIR; `distribute' calls `install' which already
depends on _SUBDIR.

De-standardize `maninstall', otherwise manpages would be installed twice.
(To be revised later.)
2002-04-23 09:03:56 +00:00
ru
f29995be3e Fixed _SUBDIR.
Obtained from:	bsd.subdir.mk
2002-04-22 09:12:18 +00:00
ru
dc24ddd41e Don't include bsd.own.mk from sys.mk, this makes it impossible
to use ``.if defined()'' inside bsd.own.mk to test for defines
in individual makefiles.  For example, setting DEBUG_FLAGS in
Makefile didn't take the desired effect on the STRIP assignment.

Added bsd.init.mk (like in NetBSD) that handles the inclusion
of ../Makefile.inc and bsd.own.mk from all bsd.*.mk files that
"build something".

Back out bsd.own.mk,v 1.15: moved OBJFORMAT initialization back
to sys.mk (several source tree makefiles want to check it early)
and removed MACHINE_ARCH initialization (it's hard to see from
looking at the commitlogs what the problem was at the time, but
now it serves no purpose).

Prohibit the direct inclusion of bsd.man.mk and bsd.libnames.mk.

Protect bsd.obj.mk from repetitive inclusion.  Prohibiting the
direct inclusion of bsd.obj.mk might be a good idea too.
2002-04-17 13:49:29 +00:00
ru
4247d565b7 Really finish the removal of ${LDDESTDIR} in bsd.lib.mk,v 1.55.
bsd.lib.mk,v 1.101 only did that partly.
2002-04-11 08:54:21 +00:00
ru
1dfb704dfa Neither .depend nor ${DEPENDFILE} depend on _SUBDIR.
(One step closer to _SUBDIR <-> _SUBDIRUSE merge.)
2002-02-06 18:08:31 +00:00
ru
82bb4e118e Fixed `objwarn' so that it doesn't issue spurious warnings,
especially now that ${.OBJDIR} is canonicalized by make(1).

Urged by:	bde
Reviewed by:	bde
2001-05-20 12:14:17 +00:00