Silicon Blade

Lesson

Recipes, Packages and Package Splitting

How one recipe can generate several packages, how files are assigned to them, and how to inspect the result

Practical Yocto Project DevelopmentUnderstanding the BuildLesson 10 of 22

Earlier, we looked at images as collections of packages rather than one large blob of software. This lesson looks at the part just before that: how a recipe’s installed files become one or more binary packages.

The distinction is worth making explicit:

recipe != package != image

A recipe describes how to build something. A package is an installable output from that recipe. An image is built by installing selected packages into a root filesystem.

That sounds simple, but many Yocto problems come from forgetting one part of the chain. A recipe may build successfully but produce a package name you did not expect. A file may be installed by do_install() but not shipped in any package. An image may include one package from a recipe but not another.

One Recipe Can Produce Several Packages

A recipe called widget-tools_1.0.bb does not have to produce only one package. It might produce:

  • widget-tools
  • widget-tools-dev
  • widget-tools-dbg
  • widget-tools-doc
  • widget-tools-utils

The main package is normally ${PN}. For a recipe named widget-tools_1.0.bb, ${PN} normally expands to:

widget-tools

The package names built from that recipe are controlled by PACKAGES, and the files assigned to each package are controlled by FILES.

A Coherent Example

Suppose we have a small project with:

  • a shared library
  • a command line tool
  • a public header
  • a small README

The source tree is not important here. The packaging behaviour is.

The recipe installs the files like this:

SUMMARY = "Widget demo library and command line tool"
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

SRC_URI = "file://widget.c file://widget.h file://widgetctl.c file://README.md"
S = "${UNPACKDIR}"

do_compile() {
    ${CC} ${CFLAGS} ${LDFLAGS} -fPIC -shared \
        -Wl,-soname,libwidget.so.1 \
        ${S}/widget.c -o libwidget.so.1
    ln -s libwidget.so.1 libwidget.so
    ${CC} ${CFLAGS} ${LDFLAGS} ${S}/widgetctl.c -L. -lwidget -o widgetctl
}

do_install() {
    install -d ${D}${libdir}
    install -m 0755 libwidget.so.1 ${D}${libdir}/libwidget.so.1
    ln -rs ${D}${libdir}/libwidget.so.1 ${D}${libdir}/libwidget.so

    install -d ${D}${bindir}
    install -m 0755 widgetctl ${D}${bindir}/widgetctl

    install -d ${D}${includedir}/widget
    install -m 0644 ${S}/widget.h ${D}${includedir}/widget/widget.h

    install -d ${D}${docdir}/${PN}
    install -m 0644 ${S}/README.md ${D}${docdir}/${PN}/README.md
}

do_install() only stages files under ${D}. It has not yet decided which binary package owns each file. That happens later during do_package.

This is a useful boundary to remember:

do_install

Copies built files into the recipe’s staged install tree, usually ${WORKDIR}/image.

do_package

Examines the staged files and splits them into binary packages using variables such as PACKAGES and FILES.

If you need a refresher on the task sequence itself, see BitBake Tasks and Classes.

Default Packages

OpenEmbedded already supplies a lot of default package splitting behaviour. That is why you often get packages such as these without writing them all yourself:

${PN}

The main runtime package. This normally contains the main binaries and runtime files.

${PN}-dev

Development files, such as headers and unversioned library links used while compiling other software.

${PN}-dbg

Debug files, where debug symbol packages are enabled.

${PN}-doc

Documentation files, such as files installed under ${docdir}.

You do not need to manually create -dev, -dbg, and -doc packages for every recipe. In normal cases, the defaults do the right thing. The skill is knowing when to inspect or adjust them.

Adding an Extra Runtime Package

Now suppose we want the command line tool to be optional. Some products need the library, but the production image may not want the widgetctl command.

We can split the command into its own package:

PACKAGES += "${PN}-tools"

FILES:${PN} += "${libdir}/libwidget.so.1"
FILES:${PN}-tools += "${bindir}/widgetctl"

This says:

  • the recipe should also emit a package called widget-tools-tools
  • the main package owns the runtime library
  • the tools package owns the command line utility

That package name is a bit ugly because the recipe itself is called widget-tools. In a real project I would probably choose a better recipe name, but the example makes the expansion obvious.

You could then install the library package without installing the command line tool:

IMAGE_INSTALL:append = " widget-tools"

Or install both:

IMAGE_INSTALL:append = " widget-tools widget-tools-tools"

Remember that IMAGE_INSTALL takes package names, not recipe names as an abstract idea. This was introduced in Images and Packages.

FILES Controls Ownership

FILES:<package> tells the packaging step which files belong in a package.

For example:

FILES:${PN}-tools += "${bindir}/widgetctl"

means the file staged at ${D}${bindir}/widgetctl should be shipped by ${PN}-tools.

The ${D} prefix is not used in FILES. FILES describes paths as they will appear on the target, such as /usr/bin/widgetctl, not their temporary location inside the recipe work directory.

Diagnosing Installed But Not Shipped

One of the most common packaging errors looks like this:

installed but not shipped

That means a file exists under ${D}, but no package in PACKAGES claimed it through its FILES patterns.

The debugging path is usually:

  1. Inspect the staged install tree under the recipe work directory.
  2. Confirm the file path that do_install() actually created.
  3. Check PACKAGES.
  4. Check the matching FILES:<package> variable.

For this example:

bitbake -e widget-tools | less

Search for:

PACKAGES=
FILES:widget-tools=
FILES:widget-tools-tools=

You can also inspect the package split output after packaging:

tmp/work/<machine>/<recipe>/<version>/packages-split/

If the file is in ${WORKDIR}/image but not in packages-split/, the problem is package assignment. If the file is not even in ${WORKDIR}/image, the problem is earlier, usually in do_install().

Runtime Dependencies Between Generated Packages

If the optional tool uses the library, the tool package should depend on the main library package:

RDEPENDS:${PN}-tools += "${PN}"

That says: if someone installs widget-tools-tools, also install widget-tools.

This is a package-level dependency. It affects installation on the target or into the root filesystem. It does not change the compile-time sysroot. The dependency lesson covers that split in more detail: Dependencies and Sysroots.

RRECOMMENDS

Sometimes a package works without another package, but the other package is a useful companion.

That is what RRECOMMENDS is for:

RRECOMMENDS:${PN} += "${PN}-tools"

This says the tools package is recommended when installing the main package, but it is not a hard runtime requirement in the same way as RDEPENDS.

Whether recommended packages end up in an image can depend on image and package manager policy. I would not use RRECOMMENDS for something the program actually needs to start. Use it for useful extras, not hidden requirements.

RPROVIDES

RPROVIDES lets a package say that it provides another runtime name.

A practical introductory case is compatibility. Suppose old images or package groups refer to widgetctl, but the new package is called ${PN}-tools. You could write:

RPROVIDES:${PN}-tools += "widgetctl"

Now package dependency resolution can treat ${PN}-tools as a provider of widgetctl.

Do not reach for RPROVIDES just to hide confusing package names. It is useful when there is a real virtual or compatibility name to provide. Otherwise, clear package names are easier to maintain.

Packagegroups

Once you split packages cleanly, you often want reusable groups of packages for images.

A packagegroup is a recipe whose job is to depend on a set of runtime packages. For example:

SUMMARY = "Widget product package group"
LICENSE = "MIT"

inherit packagegroup

RDEPENDS:${PN} = " \
    widget-tools \
    widget-tools-tools \
"

An image can then install the packagegroup:

IMAGE_INSTALL:append = " packagegroup-widget-product"

This keeps image recipes cleaner, especially once a product has dozens of packages. The image says what role it is building. The packagegroup carries the software set for that role.

Identifying Which Package Owns a File

After a package has been built, oe-pkgdata-util is the practical tool for asking packaging questions.

To find which package owns a file:

oe-pkgdata-util find-path /usr/bin/widgetctl

To list the files in a package:

oe-pkgdata-util list-pkg-files widget-tools-tools

To see which recipe produced a package:

oe-pkgdata-util lookup-recipe widget-tools-tools

These commands use pkgdata generated by the build, so they are most useful after the relevant recipe or image has been built.

Summary

Package splitting is where the recipe/package/image distinction becomes real.

The main ideas from this lesson are:

  • a recipe can generate several binary packages
  • ${PN} is normally the main package name
  • PACKAGES lists the packages a recipe emits
  • FILES:<package> decides which staged files belong to which package
  • do_install() stages files, while do_package splits them into packages
  • RDEPENDS can express runtime dependencies between generated packages
  • RRECOMMENDS is for useful but non-essential runtime companions
  • RPROVIDES gives a package an additional runtime provider name
  • packagegroups are useful for reusable image package sets
  • oe-pkgdata-util is often the fastest way to find which package owns a file

Check your understanding

Quick quiz: package splitting

Check the relationship between recipes, packages, and installed files.

Question 1Which statement is the safest Yocto mental model?
Question 2A file is installed by `do_install()` but triggers an installed-but-not-shipped error. What should you inspect first?
Question 3What does `${PN}` normally represent?