Lesson

Working with Variables

How to set, override, and manipulate variables in the Yocto Project

The Yocto ProjectCreating The SystemLesson 6 of 16

Variables are one of the most important parts of working with the Yocto Project. They control almost everything:

where files are installed
which packages are built
how recipes behave
how images are configured
how machine and distro specific behaviour is selected

If you are comfortable with variables, the rest of Yocto starts to make much more sense.

Why Variables Matter

BitBake metadata is mostly about describing build behaviour through variables.

A recipe does not have to hard-code paths such as /usr/bin or /etc. Instead, they often makes use of variables such as ${bindir} and ${sysconfdir}.

Variable types

There are two types of variables:

Global Variables
- Defined in configuration files (.conf)
- By convention in uppercase, e.g. MACHINE
Local Variables
- Defined in recipes (.bb and .bbappend)

Variables can be assigned and changed in many different ways. This makes the system flexible, but it also means you need to understand:

how to assign a value
when a value is expanded
how to add to or remove from an existing value
how overrides affect the final result

Common Assignment Operators

The most common operators are:

Setting a Value

Set a variable absolutely. BitBake will take the value assigned at that point in time. The only way to change it would be to use another absolute assignment in a later recipe.

VAR = "value"

Set a value but only if the variable has not previously been set somewhere else. This is useful for default values, where the user may want to override it in their own recipe.

Use ?= when you want to supply a default only if nobody has already provided a value. This is more useful if you are writing recipes that may be shared and used in different projects.

VAR ?= "value"

??=

Set a weak default value that may still be replaced later. This also sets a default value, but waits until the end of the processing to commit it. This means that a later recipe can also override the value with either = or ?=

VAR ??= "value"

Multi-line Assignments

You can assign a variable value that goes over multiple lines by escaping the ‘new-line’. This is often used to make the assignment easier to read.

VAR = "This value \
  needs multiple lines \
  to make it more readable \
"

Changing Values

Appending values

This is where it can get really confusing as there seem to be so many ways to do the same thing!

These are the options:

:append: Append text to the final value of a variable. It does not automatically add a space - you will need to do that
+=: Append text to the final value of a variable. This will automatically add a space.
:prepend: Prepend text to the final value of a variable.It does not automatically add a space*
=+: Prepend text to the final value of a variable. This will automatically add a space

* If you noticed, yes, that was deliberate.

To summarise:

Operation	Parse or expansion time	Automatic space	Typical use
`:append`	Expansion	No	Safely append
`+=`	Parse	Yes	Add items immediately
`:prepend`	Expansion	No	Safely prepend
`=+`	Parse	Yes (prepend)	Prepend items immediately

Automatic Space?

The contents of many variables in a Yocto project are a list of things separated by whitespace. Bitbake will then split the string into the individual tokens.

However, there are some variables, such as paths, that use a colon, :, as the separator, so these should not have additional whitespace. This is why there are options not to automatically add the space.

Parse or Expansion time?

What this means is when does the append/prepend actually take place.

At parse time means when BitBake reads the variable. At expansion time means when BitBake uses it.

FOO += "bar"

This modifies the variable immediately.

For example:

FOO = "a"
FOO += "b"

Result:

FOO = "a b"

because += inserts a space automatically. However, if someone later does

FOO = "c"

the previous append is lost.

Example:

FOO =  "a"
FOO += "b"
FOO =  "c"

Final value:

FOO = "c"

When you use

FOO:append = " bar"

This doesn’t modify the variable immediately.

Instead it records”

“When someone finally asks for FOO, append ' bar'.”

Removing values

At least this is much simpler - there is only the one main option:

:remove: Remove matching text from the final value of a variable.

The Yocto Project documentation generally encourages using the modern override syntax (:append, :prepend, :remove) because it is deterministic and avoids subtle ordering problems that occur with operators like += and =+.

Practical Examples

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

This pattern is common in .bbappend files when extending the search path for additional files. Note, in this case, we absolutely do not what any extra spaces, as that would mess up the path.

Removing Values

Use :remove when you need to remove something from a value that has already been built up elsewhere.

DISTRO_FEATURES:remove = "x11"
IMAGE_INSTALL:remove = "nano"

This is much cleaner than trying to reconstruct the whole variable by hand.

Immediate Expansion with `:=`

Normally, many variables are expanded later when BitBake actually needs them. Sometimes you want the right-hand side to be expanded immediately.

FILESEXTRAPATHS:prepend := "${THISDIR}/files:"

This is important because THISDIR depends on the file being parsed. If you wait until later expansion, you may no longer get the value you expected.

Recipe Work Directories

When you start reading real recipes, a small set of internal path variables turns up again and again.

They are not usually variables you create yourself. They are the directories BitBake prepares for a recipe while it moves through the normal tasks:

fetch
unpack
patch
configure
compile
install
package

The exact paths depend on your machine, distro, recipe name, and version, but a normal target recipe will usually live under something like:

tmp/work/<target-system>/<recipe>/<version>/

For a recipe called hello-recipe_1.0.bb, that might be:

tmp/work/core2-64-poky-linux/hello-recipe/1.0/

The Main Directories

WORKDIR: The recipe’s private work directory. It is the parent area where BitBake keeps temporary files, unpacked sources, build output, install staging, logs, and packaging output for one recipe.
UNPACKDIR: The directory where do_unpack places fetched source files. In modern OpenEmbedded-Core, this normally points to ${WORKDIR}/sources.
S: The source directory used by tasks such as do_patch, do_configure, and do_compile. By default, it is normally ${UNPACKDIR}/${BP}.
B: The build directory used by configure and compile tasks. By default, it is the same as ${S}, but classes such as cmake often use a separate build directory.
D: The destination directory used by do_install(). Files staged here are later split into packages. It normally points to ${WORKDIR}/image.

For the same hello-recipe_1.0.bb example, the important paths would normally look like this:

${WORKDIR}
  tmp/work/core2-64-poky-linux/hello-recipe/1.0

${UNPACKDIR}
  tmp/work/core2-64-poky-linux/hello-recipe/1.0/sources

${S}
  tmp/work/core2-64-poky-linux/hello-recipe/1.0/sources/hello-recipe-1.0

${B}
  usually the same as ${S}, unless a class or recipe changes it

${D}
  tmp/work/core2-64-poky-linux/hello-recipe/1.0/image

The value ${BP} is usually ${BPN}-${PV}. For hello-recipe_1.0.bb, that means:

PN  = hello-recipe
BPN = hello-recipe
PV  = 1.0
BP  = hello-recipe-1.0

PN is the recipe name, including special forms such as -native or multilib prefixes when they apply. BPN is the base recipe name with those special parts removed. BP combines the base name and version.

Why `${S}` Sometimes Needs Setting

The default ${S} value works when the unpacked source directory matches ${BP}.

For example, this archive layout is easy:

hello-recipe-1.0.tar.gz
  hello-recipe-1.0/
    Makefile
    hello-recipe.c

The default source directory points to:

${UNPACKDIR}/hello-recipe-1.0

If the source is unpacked somewhere else, set ${S} to match reality. For example, this recipe asks the Git fetcher to unpack into a directory called git:

SRC_URI = "git://github.com/example/hello-recipe.git;branch=main;protocol=https;destsuffix=git"
SRCREV = "0123456789abcdef0123456789abcdef01234567"

S = "${UNPACKDIR}/git"

That tells configure, compile, and patch tasks where the source tree actually is.

Without destsuffix=git, modern OpenEmbedded-Core normally unpacks Git sources under ${UNPACKDIR}/${BP}, so you often do not need to set ${S} for a simple Git recipe.

For recipes made only from local files, there may not be a versioned source directory at all. In that case, it is common to use ${UNPACKDIR} directly:

SUMMARY = "Small local shell tool"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

SRC_URI = "file://hello-tool"

S = "${UNPACKDIR}"

do_install() {
    install -d ${D}${bindir}
    install -m 0755 ${S}/hello-tool ${D}${bindir}/hello-tool
}

do_unpack stages hello-tool under ${UNPACKDIR}, ${S} points there, and do_install() copies the file into ${D}${bindir}.

How `${B}` and `${D}` Fit In

${B} matters most when the build system keeps generated files away from the source tree.

A CMake recipe is a common example:

SUMMARY = "Small CMake application"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

SRC_URI = "git://github.com/example/hello-cmake.git;branch=main;protocol=https;destsuffix=git"
SRCREV = "0123456789abcdef0123456789abcdef01234567"

inherit cmake

S = "${UNPACKDIR}/git"

The cmake class can configure from ${S} and build in ${B}. You normally do not need to call CMake yourself. The recipe mostly tells the class where the source is and which options to use.

${D} belongs to installation and packaging:

do_install:append() {
    install -d ${D}${sysconfdir}/hello-cmake
    install -m 0644 ${UNPACKDIR}/hello-cmake.conf ${D}${sysconfdir}/hello-cmake/hello-cmake.conf
}

That installs into the temporary package image, not into /etc on your build host. Later, do_package reads ${D} and splits its contents into packages.

Other useful directories under ${WORKDIR} include:

${T}, usually ${WORKDIR}/temp, for task logs and generated run scripts
${RECIPE_SYSROOT}, for target headers and libraries from build-time dependencies
${RECIPE_SYSROOT_NATIVE}, for native tools used while building
${WORKDIR}/packages-split, where do_package writes the split package contents

You do not usually write to these directly, but they are very useful when debugging.

BitBake File Search Paths

SRC_URI can refer to remote sources, archives, Git repositories, local files, and patches.

Local files use the file:// form:

SRC_URI = " \
    file://hello-tool \
    file://hello-tool.conf \
    file://0001-fix-startup-message.patch \
"

BitBake then has to answer a simple question:

Where on disk is hello-tool.conf?

That is what FILESPATH is for.

`FILESPATH`

FILESPATH is a colon-separated list of directories that BitBake searches when it sees a file:// entry in SRC_URI.

You normally do not set FILESPATH directly. The base metadata builds it for you from recipe-related locations. The default pattern is based on:

${FILE_DIRNAME}/${BP}
${FILE_DIRNAME}/${BPN}
${FILE_DIRNAME}/files

${FILE_DIRNAME} is the directory containing the recipe file being processed. For this recipe:

meta-my-software/recipes-apps/hello-tool/hello-tool_1.0.bb

the default recipe-related search locations include:

meta-my-software/recipes-apps/hello-tool/hello-tool-1.0/
meta-my-software/recipes-apps/hello-tool/hello-tool/
meta-my-software/recipes-apps/hello-tool/files/

That is why this recipe can find files in its own files/ directory without extra configuration:

meta-my-software/
  recipes-apps/
    hello-tool/
      hello-tool_1.0.bb
      files/
        hello-tool
        hello-tool.conf

SRC_URI = " \
    file://hello-tool \
    file://hello-tool.conf \
"

BitBake searches the directories in FILESPATH, finds the matching local files, then do_unpack places them under ${UNPACKDIR}.

FILESPATH is also affected by file overrides such as machine and distro overrides. That allows patterns like machine-specific defconfig files, but you do not need to master that immediately. For everyday recipe work, remember that ${BP}, ${BPN}, and files are the normal recipe-local directories.

`FILESEXTRAPATHS` in `.bbappend` Files

The default recipe search paths are based on the original .bb file’s directory. They do not automatically include the directory containing your .bbappend in a different layer.

That is why appends commonly add to FILESEXTRAPATHS:

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

This line has three important parts:

:prepend: Put your directory at the start of the search path so your project files are found before matching files from the original recipe layer.
:=: Expand the right-hand side immediately while BitBake is parsing this append. This matters because ${THISDIR} means “the directory of the file currently being parsed”.
trailing `:`: Keep the value as a colon-separated path list. Without the trailing colon, your new path can be joined to the next path and become one invalid path.

${THISDIR} is different from ${FILE_DIRNAME} in the practical case that matters most:

in the original recipe, it points at the recipe directory while the recipe is parsed
in your .bbappend, it points at your append directory while the append is parsed

That makes it useful for adding files stored beside an append.

Complete Search Example

Suppose an upstream layer contains:

meta-vendor/
  recipes-apps/
    hello-tool/
      hello-tool_1.0.bb
      files/
        hello-tool

The original recipe says:

SRC_URI = "file://hello-tool"
S = "${UNPACKDIR}"

Your project layer adds a configuration file:

meta-my-distro/
  recipes-apps/
    hello-tool/
      hello-tool_%.bbappend
      hello-tool/
        hello-tool.conf

The append says:

FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

SRC_URI:append = " file://hello-tool.conf"

do_install:append() {
    install -d ${D}${sysconfdir}/hello-tool
    install -m 0644 ${UNPACKDIR}/hello-tool.conf ${D}${sysconfdir}/hello-tool/hello-tool.conf
}

For hello-tool_1.0.bb, ${PN} is hello-tool. While parsing the append, ${THISDIR} is:

meta-my-distro/recipes-apps/hello-tool

So the prepended search path is:

meta-my-distro/recipes-apps/hello-tool/hello-tool

When BitBake sees:

file://hello-tool.conf

it searches the path list and finds:

meta-my-distro/recipes-apps/hello-tool/hello-tool/hello-tool.conf

After do_unpack, the file is available to tasks at:

${UNPACKDIR}/hello-tool.conf

Then do_install:append() stages it into:

${D}${sysconfdir}/hello-tool/hello-tool.conf

which becomes:

/etc/hello-tool/hello-tool.conf

on the target once the package is installed into an image.

Overrides

Overrides let you set different values depending on context such as:

the package
the machine
the distro
the recipe task

For example:

SRC_URI:append:qemux86 = " file://qemu-extra.cfg"
IMAGE_INSTALL:append:pn-core-image-minimal = " strace"

This means the change only applies when that specific override is active.

Overrides are one of the most powerful parts of the Yocto Project, but also one of the easiest places to create confusing behaviour if you forget the context in which the variable is being evaluated. THis will be covered in more detail in a later chapter.

Basic Expansion

Variables are usually referenced using ${...} syntax.

MYDIR = "/opt/demo"
MYFILE = "${MYDIR}/config.txt"

In that example, MYFILE expands to /opt/demo/config.txt.

You will see this style everywhere in Yocto metadata.

Variable Expansion in Shell Tasks

Inside shell tasks, BitBake variables are expanded before the shell runs.

do_install() {
    install -d ${D}${bindir}
    install -m 0755 mytool ${D}${bindir}/mytool
}

In that example:

${D} is the destination staging area
${bindir} is usually /usr/bin

So the file is installed into the package staging area under /usr/bin.

Inline Python Expansion

Sometimes the value you need depends on logic rather than a fixed string.

You can use the ${@...} to evaluate a Python expression.

An example of the type of expression commonly seen:

ROOTFS_DEVICE = "${@'/dev/mmcblk0' if d.getVar('IMAGE_BASENAME') \
  == 'dev-image' else '/dev/mmcblk1'}"

This makes use of the Python syntax

VALUE equals X if (thing is true) else VALUE equals Y

Notes:

d.getVar() reads a BitBake variable
the expression returns the string that becomes the final value

Use this carefully. It is useful, but large amounts of inline Python can make metadata hard to read.

Order of Evaluation

Common Patterns

Add a Package to an Image

IMAGE_INSTALL:append = " i2c-tools"

Add a Distro Feature

DISTRO_FEATURES:append = " systemd"

Remove a Distro Feature

DISTRO_FEATURES:remove = "x11"

Set a Package-Specific Variable

RDEPENDS:${PN} += "bash"

${PN} means “the package name for this recipe”.

Add Files to a Package

FILES:${PN} += " /usr/local/bin/mytool"

This is a common pattern when a recipe installs something outside the usual default packaging locations.

Common Mistakes

Forgetting the Leading Space

This is one of the most common mistakes:

IMAGE_INSTALL:append = "htop"

That will usually produce a broken combined value.

The correct form is:

IMAGE_INSTALL:append = " htop"

Overwriting Instead of Extending

If you write:

DISTRO_FEATURES = "systemd"

you replace the whole variable.

That may be what you want, but often what you really meant was:

DISTRO_FEATURES:append = " systemd"

Using Immediate Expansion Without Needing It

:= is useful, but do not use it everywhere. If you do not need immediate expansion, prefer the simpler form.

How to Inspect Variable Values

One of the most useful debugging commands is:

bitbake-getvar <variable-name>

For example:

bitbake-getvar DISTRO_FEATURES
NOTE: Starting bitbake server...
#
# $DISTRO_FEATURES [6 operations]
#   set? /home/ming/Projects/demos/YoctoTraining/build/../layers/project/meta-ecw/meta-ecw-core/conf/distro/microforge.conf:20
#     "${DISTRO_FEATURES_DEFAULT} ${MICROFORGE_DEFAULT_DISTRO_FEATURES}"
#   :remove /home/ming/Projects/demos/YoctoTraining/build/../layers/project/meta-ecw/meta-ecw-core/conf/distro/microforge.conf:21
#     "x11"
#   set? /home/ming/Projects/demos/YoctoTraining/build/../layers/third-party/poky/meta/conf/distro/include/default-distrovars.inc:29
#     "${DISTRO_FEATURES_DEFAULT}"
#   :append /home/ming/Projects/demos/YoctoTraining/build/../layers/third-party/poky/meta/conf/distro/include/init-manager-systemd.inc:2
#     " systemd usrmerge"
#   set /home/ming/Projects/demos/YoctoTraining/build/../layers/third-party/poky/meta/conf/documentation.conf:145
#     [doc] "The features enabled for the distribution."
#   set? /home/ming/Projects/demos/YoctoTraining/build/../layers/third-party/poky/meta/conf/bitbake.conf:903
#     ""
# pre-expansion value:
#   "${DISTRO_FEATURES_DEFAULT} ${MICROFORGE_DEFAULT_DISTRO_FEATURES} systemd usrmerge"
DISTRO_FEATURES="acl alsa bluetooth debuginfod ext2 ipv4 ipv6 pcmcia usbgadget usbhost wifi xattr nfs zeroconf pci 3g nfc  vfat seccomp usrmerge systemd systemd-resolved networkd  systemd usrmerge"

This will show you not only the final value of variable, but also, importantly, which file(s) set the values.

If a variable is not behaving the way you expect, bitbake-getvar is usually the best place to start.

Summary

use = for normal assignment
use ?= and ??= for defaults
use :append, :prepend, and :remove to manipulate existing values
use := only when immediate expansion is required
remember that overrides change values depending on context
use bitbake-getvar to inspect the final value when debugging

Question 1You want to add `htop` to an existing image package list without replacing what is already there. Which form is most appropriate?

`IMAGE_INSTALL = "htop"``IMAGE_INSTALL:append = " htop"``IMAGE_INSTALL ??= "htop"`

Question 2What is the actual problem with writing `IMAGE_INSTALL:append = "htop"` for a space-separated list?

BitBake rejects appended values unless they start with a spaceBitBake does not insert the separator for you, so the new item can run into the previous valueBitBake treats the append as an override if the string starts immediately

Question 3Which case is the best reason to use `:=` instead of normal deferred expansion?

When you want the value resolved immediately because it depends on parse-time context like `THISDIR`When you want to remove one item from a final list valueWhen you want later overrides to have more effect than usual

Why Variables Matter

Variable types

Common Assignment Operators

Setting a Value

Multi-line Assignments

Changing Values

Appending values

Automatic Space?

Parse or Expansion time?

Removing values

Practical Examples

Removing Values

Immediate Expansion with :=

Recipe Work Directories

The Main Directories

Why ${S} Sometimes Needs Setting

How ${B} and ${D} Fit In

BitBake File Search Paths

FILESPATH

FILESEXTRAPATHS in .bbappend Files

Complete Search Example

Overrides

Basic Expansion

Variable Expansion in Shell Tasks

Inline Python Expansion

Order of Evaluation

Common Patterns

Add a Package to an Image

Add a Distro Feature

Remove a Distro Feature

Set a Package-Specific Variable

Add Files to a Package

Common Mistakes

Forgetting the Leading Space

Overwriting Instead of Extending

Using Immediate Expansion Without Needing It

How to Inspect Variable Values

Summary

Quick quiz: Yocto variables

Immediate Expansion with `:=`

Why `${S}` Sometimes Needs Setting

How `${B}` and `${D}` Fit In

`FILESPATH`

`FILESEXTRAPATHS` in `.bbappend` Files