How does Shared State Cache in Yocto work?

Question

Can any one explain to me more comprehensively how the sstate cache works in yocto?

This explanation is far from clear.

I don't understand when such situation occurs:

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Running setscene task 118 of 155 (virtual:native:/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/pseudo/pseudo_git.bb:do_populate_sysroot_setscene)
NOTE: Running setscene task 119 of 155 (/home/lulianhao/poky-build/edwin/poky/meta/recipes-devtools/quilt/quilt-native_0.48.bb:do_populate_sysroot_setscene)

When it found the artifacts or got the candidates, and then checks the signatures. I want to know when setscene tasks are actually run.

Additional question: When it looks in local sstate_cache folder and when into mirror?

Answer 1

The Yocto Project manual has a section devoted to Shared State Cache.

To answer your question, the sstate-cache folder is checked first, then the mirrors are checked if nothing is found locally.

This cache is built based on a set of inputs which are hashed into "signatures" which can be found in the $BUILD_DIR/tmp/stamps, but keep in mind you'll need bitbake-dumpsigs to view the file. Having a look at bitbake-dumpsigs and bitbake-diffsigs can help you understand how the cache works. Also there is a great "Tips & Tricks" article on Understanding What Changed in your build environment.

While it can take some time to understand, shared state cache is extremely valuable and rigorously tested.

In terms of tracing dependancies, for example why your image might contain passwd, bitbake -g will give you a dependency tree and oe-pkgdata-util find-path can help you understand which recipe resulted in a given binary on the resultant image.

Useful tip: "When we need to rebuild from scratch, we either remove the build/tmp so that we can use sstate-cache to speed up the build or we remove both build/tmp and sstate-cache so that no cache is reused during the build."¹

_{¹Salvador, Otavio, and Daiane Angolini. "6.2 Understanding Shared State Cache." Embedded Linux Development with Yocto Project}