diff options
author | Bryan Drewery <bdrewery@FreeBSD.org> | 2015-10-17 03:51:50 +0000 |
---|---|---|
committer | Bryan Drewery <bdrewery@FreeBSD.org> | 2015-10-17 03:51:50 +0000 |
commit | fb84a99e0f975a86d6e6ae98923192648f6bd86e (patch) | |
tree | d3c48adfae08869a3e059f76affcf8324aa0d75d | |
parent | 23ffbc10116222c77d52dcece6c9bbca424b5120 (diff) | |
download | src-fb84a99e0f975a86d6e6ae98923192648f6bd86e.tar.gz src-fb84a99e0f975a86d6e6ae98923192648f6bd86e.zip |
Rework the world subdir build targets to use the standard SUBDIR_PARALLEL mechanism.
Back in r30113, the 'par-*' targets were added to parallelize portions of
the build in a very similar fashion as the SUBDIR_PARALLEL feature used in
r263778. Calling a target without 'par-' (for 'parallel') resulted in the
standard bsd.subdir.mk handling without parallelization. Given we have
SUBDIR_PARALLEL now there is no reason to duplicate the handling here.
In build logs this will result in the ${dir}.${target}__D targets now showing
as the normal ${target}_subdir_${dir} targets.
I audited all of the uses of Makefile.inc1 and Makefile's targets that use
bsd.subdir.mk and found that all but 'all' and 'install' were fine to use
as always parallel.
- For 'install' (from installworld -j) the ordering of lib/ and libexec/
before the rest of the system (described in r289433), and etc/ being last
(described in r289435), is all that matters. So now a .WAIT is added in
the proper places when invoking any 'install*' target. A parallel
installworld does work and took 46% of the time a non-parallel
install would take on my system with -j15 to ZFS.
- For 'all' I left the default handling for this to not run in parallel. A
'par-all' target is still used by the 'everything' stage of buildworld
to continue building in parallel as it already has been. This works
because most of the dependencies are handled by the early bootstrap
phases as well as 'libraries' and 'includes' phases. This lets
all of the SUBDIR build in parallel fine, such as bin/ and lib/. This
will not work if the user invokes 'all' though as we have dependencies
spread all over the system with no way to depend between them (except
for the dirdeps feature in the META_MODE build). Calling 'make all'
from the top-level is still useful at least when using SUBDIR_OVERRIDE.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Notes
Notes:
svn path=/head/; revision=289438
-rw-r--r-- | Makefile.inc1 | 47 |
1 files changed, 25 insertions, 22 deletions
diff --git a/Makefile.inc1 b/Makefile.inc1 index 2cfea9e422a7..e491053e2f1d 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -63,6 +63,10 @@ SRCDIR?= ${.CURDIR} SUBDIR= ${SUBDIR_OVERRIDE} .else SUBDIR= lib libexec +.if make(install*) +# Ensure libraries are installed before progressing. +SUBDIR+=.WAIT +.endif SUBDIR+=bin .if ${MK_CDDL} != "no" SUBDIR+=cddl @@ -115,6 +119,9 @@ SUBDIR+= ${_DIR} # by calling 'makedb' in share/man. This is only relevant for # install/distribute so they build the whatis file after every manpage is # installed. +.if make(install*) +SUBDIR+=.WAIT +.endif SUBDIR+=etc .if defined(NOCLEAN) @@ -589,9 +596,9 @@ _cleanobj: @echo "--------------------------------------------------------------" @echo ">>> stage 2.1: cleaning up the object tree" @echo "--------------------------------------------------------------" - ${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR:S/^/par-/} + ${_+_}cd ${.CURDIR}; ${WMAKE} ${CLEANDIR} .if defined(LIB32TMP) - ${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 ${CLEANDIR:S/^/par-/} + ${_+_}cd ${.CURDIR}; ${LIB32WMAKE} -f Makefile.inc1 ${CLEANDIR} .endif .endif _obj: @@ -599,7 +606,7 @@ _obj: @echo "--------------------------------------------------------------" @echo ">>> stage 2.2: rebuilding the object tree" @echo "--------------------------------------------------------------" - ${_+_}cd ${.CURDIR}; ${WMAKE} par-obj + ${_+_}cd ${.CURDIR}; ${WMAKE} obj _build-tools: @echo @echo "--------------------------------------------------------------" @@ -618,7 +625,7 @@ _includes: @echo "--------------------------------------------------------------" @echo ">>> stage 4.1: building includes" @echo "--------------------------------------------------------------" - ${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks par-includes + ${_+_}cd ${.CURDIR}; ${WMAKE} SHARED=symlinks includes _libraries: @echo @echo "--------------------------------------------------------------" @@ -632,7 +639,7 @@ _depend: @echo "--------------------------------------------------------------" @echo ">>> stage 4.3: make dependencies" @echo "--------------------------------------------------------------" - ${_+_}cd ${.CURDIR}; ${WMAKE} par-depend + ${_+_}cd ${.CURDIR}; ${WMAKE} depend everything: @echo @echo "--------------------------------------------------------------" @@ -1944,22 +1951,18 @@ _startup_libs: ${_startup_libs:S/$/__L/} _prebuild_libs: ${_prebuild_libs:S/$/__L/} _generic_libs: ${_generic_libs:S/$/__L/} -.for __target in all clean cleandepend cleandir depend includes obj -.for entry in ${SUBDIR} -${entry}.${__target}__D: .PHONY .MAKE - ${_+_}@set -e; if test -d ${.CURDIR}/${entry}.${MACHINE_ARCH}; then \ - ${ECHODIR} "===> ${DIRPRFX}${entry}.${MACHINE_ARCH} (${__target})"; \ - edir=${entry}.${MACHINE_ARCH}; \ - cd ${.CURDIR}/$${edir}; \ - else \ - ${ECHODIR} "===> ${DIRPRFX}${entry} (${__target})"; \ - edir=${entry}; \ - cd ${.CURDIR}/$${edir}; \ - fi; \ - ${MAKE} ${__target} DIRPRFX=${DIRPRFX}$${edir}/ -.endfor -par-${__target}: ${SUBDIR:S/$/.${__target}__D/} -.endfor +# Enable SUBDIR_PARALLEL when not calling 'make all', unless called as +# 'par-all'. This is because it is unlikely that running 'make all' from +# the top-level, especially with a SUBDIR_OVERRIDE or LOCAL_DIRS set, +# will have a reliable build if SUBDIRs are built in parallel. This is +# safe for the world stage of buildworld though since it has already +# built libraries in a proper order and installed includes into WORLDTMP. +# Special handling is done for SUBDIR ordering for 'install*' to avoid +# trashing a system if it crashes mid-install. +par-all: all .PHONY +.if !make(all) +SUBDIR_PARALLEL= +.endif .include <bsd.subdir.mk> @@ -2267,7 +2270,7 @@ _xi-cross-tools: .endfor _xi-includes: - ${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 par-includes \ + ${_+_}cd ${.CURDIR}; ${CD2MAKE} -f Makefile.inc1 includes \ DESTDIR=${XDDESTDIR} _xi-libraries: |