Page MenuHome

Add option to use ccache for build if available
ClosedPublic

Authored by Matt Hill (theothermatt) on Nov 29 2020, 7:56 PM.

Details

Summary

This adds an option (USE_CCACHE) to build using ccache if found. Makefiles, ninja and Xcode are supported. I'm afraid I have no knowledge of Windows development, so I don't know if this could be made to work with Visual Studio too.

This is all based on https://github.com/TheLartians/Ccache.cmake

Examples:

From the root of the blender source tree, run make BUILD_CMAKE_ARGS=-DUSE_CCACHE=YES ninja to build with ninja using ccache.

cmake . -DUSE_CCACHE=YES -B ../build_xcode -G "Xcode" will build an Xcode project which uses ccache.

Diff Detail

Repository
rB Blender
Branch
arcpatch-D9665 (branched from master)
Build Status
Buildable 11577
Build 11577: arc lint + arc unit

Event Timeline

Matt Hill (theothermatt) requested review of this revision.Nov 29 2020, 7:56 PM
Matt Hill (theothermatt) created this revision.
Matt Hill (theothermatt) edited the summary of this revision. (Show Details)
Ankit Meel (ankitm) requested changes to this revision.EditedNov 30 2020, 10:13 AM

Thank you for working on it.

This adds an option (USE_CCACHE) to build using ccache if found.

In line with other such options, WITH_COMPILER_CCACHE could be better.

Makefiles, ninja and Xcode are supported.

It is trivial to use ccache with ninja and makefile using :

CMAKE_C_COMPILER_LAUNCHER=ccache
CMAKE_CXX_COMPILER_LAUNCHER=ccache

documented at https://wiki.blender.org/wiki/Building_Blender/Options#Setup_For_Developers and https://cmake.org/cmake/help/latest/prop_tgt/LANG_COMPILER_LAUNCHER.html?highlight=ccache

I'm afraid I have no knowledge of Windows development, so I don't know if this could be made to work with Visual Studio too.

On windows, they use sccache and from what I see in chat, seems to be working fine for people who use it.

As for the approach, it seems long winded and outdated. Since Xcode adds so many more flags to the compiler than Ninja does, there's no point trying to keep the flags same for both generators (to preserve the compiler cache).
I have an initial cache file you can see at F9413084 which allows me to set variables and inject compiler/link flags which I cannot do otherwise without modifying source code.
Only lines 9-12 are enough to make ccache work on Xcode. Please try something similar to that.

EDIT:
One way would be to use execute_process(.. cmake -E create_symlink ccache BUILD_DIRECTORY/ccache_launcher) and then set XCODE..CC to this newly created ccache_launcher and similar for CXX and LD and LDPLUSPLUS

This revision now requires changes to proceed.Nov 30 2020, 10:13 AM

@Ankit Meel (ankitm) thanks for taking a look at this so quickly.

Changes as requested:

  1. Renamed option to WITH_COMPILER_CCACHE
  1. Taken your suggestion to use cmake -E create_symlink to generate links in the build directory to the ccache binary, named according to the compiler - ccache figures out which compiler to use based on that. You can't just link to ccache because it appears CMake's Xcode generator doesn't include the compiler as the first argument.
  1. Moved everything I can to the platform specific files

Thanks again for looking at this - I'm new to CMake and Blender, so I appreciate it.

Ankit Meel (ankitm) requested changes to this revision.EditedDec 2 2020, 9:59 PM

Thanks for taking care of the details!

You can't just link to ccache because it appears CMake's Xcode generator doesn't include the compiler as the first argument.

Oh I see. The commands can be seen in Xcode's build log tab after expanding any compile unit. If it has our custom clang/clang++ as the executable, ccache will work.

Moved everything I can to the platform specific files

The duplicated code for makefiles and Ninja could be in main file too, but don't change it until other reviewers comment.

Final note: I cannot see what's around the changed lines, so create the diff using git diff -U100 or copy it using git diff -U100 | pbcopy

I'll test it on the next update.

build_files/cmake/platform/platform_apple.cmake
478

This is verbose.. a lot of variables are set quietly and that is assumed as success.
Also, when using Xcode one sees the following:

Skipping setup of Ccache for makefiles/ninja since using Xcode generator
finding ccache
finding ccache - found // or something to that effect.
Setting up Ccache for Xcode: ${CCACHE_PROGRAM}

Report only if something fails.

484

Also turn the option off WITH_COMPILER_CCACHE.
See other warnings in the same file that are printed if a library is not found.
Nitpick not-> NOT

487

same note for verbosity.

build_files/cmake/platform/platform_apple_xcode.cmake
165

Nitpick that Ccache stylisation has not been used here. (;

166

simlink -> symlink

build_files/cmake/platform/platform_unix.cmake
691

Same note for verbosity.

697

Same note to disable the option.

This revision now requires changes to proceed.Dec 2 2020, 9:59 PM
build_files/cmake/platform/platform_apple_xcode.cmake
168

COMMAND cmake -E .. -> COMMAND ${CMAKE_COMMAND} -E .. .

@Ankit Meel (ankitm) hopefully this addresses all your comments with the previous revision:

  • Removed a lot of the verbosity in the output
  • Turn off WITH_COMPILER_CCACHE if Ccache can't be found
  • Updated "ccache not found" message to match other messages for libraries not found
  • Fixed typo
  • Use ${CMAKE_COMMAND} instead of cmake in execute_process()
  • Move the new option near other options. Fix typos. Use temporary variable to reduce repetition.

Code works, tested with Ninja and Xcode
Seeking input on location of these code blocks especially the duplicated makefiles and ninja code for unix and macOS.

Looks fine if my comment is addressed.

Would also be nice if a ccache target was added to GNUmakefile to match the Windows.

build_files/cmake/platform/platform_apple_xcode.cmake
167–168

Can we put these in a ${CMAKE_BINARY_DIR}/ccache directory?

To avoid potential naming conflict, and to make it clear what this is when looking in the build folder.

@Brecht Van Lommel (brecht) this adds a ${CMAKE_BINARY_DIR}/ccache directory to put the symlinks in, and adds a target to GNUmakefile.

Do you need anything else from me for this patch? I saw @Brecht Van Lommel (brecht) had accepted it so I assumed not, but just let me know.

Ah I shouldn't have added both as blocking.. I'll commit it tomorrow.

This revision was not accepted when it landed; it landed in state Needs Review.Dec 21 2020, 6:19 AM
This revision was automatically updated to reflect the committed changes.