Silicon Blade

Back to Articles

Articles

Golang Recipes in a Yocto Project

Yocto supports Golang recipes, but they are slightly different to standard ones.

YoctoPublished 25/04/20245 min read
YoctoGolang
Golang Recipes in a Yocto Project

Yocto supports Go recipes, but Go modules make them a little different from a normal application recipe.

The important rule is the same as for every other Yocto recipe: fetching belongs in do_fetch(), not in do_compile().

That matters because a normal upstream Go workflow is quite happy to let go build download module dependencies into the module cache. That is fine on a developer laptop. It is not fine in a reproducible Yocto build, where the build should be able to run from known sources, from DL_DIR, and with network access disabled after fetching.

A Small Go Module

Here is a small application, based on the hello-go example from the Yocto training material, but with one real module dependency added.

The source tree looks like this:

hello-go/
  go.mod
  go.sum
  cmd/
    hello-go/
      main.go

go.mod defines the module path and the module dependencies:

module example.com/hello-go

go 1.22

require github.com/fatih/color v1.18.0

cmd/hello-go/main.go:

package main

import "github.com/fatih/color"

func main() {
	color.Green("hello-go: built by BitBake with Go modules")
}

In normal Go development, you would run:

go mod tidy
go build ./cmd/hello-go

The Go tool would use go.mod and go.sum, contact a module proxy or VCS server if needed, and populate the module cache.

In Yocto, that network fetching has to be represented in the recipe metadata instead.

The Recipe

For a Wrynose-era Go module recipe, start with go-mod:

SUMMARY = "Small Go module example"
DESCRIPTION = "Builds a small Go application using Yocto Go module support."
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"

GO_IMPORT = "example.com/hello-go"
GO_INSTALL = "${GO_IMPORT}/cmd/hello-go"

SRC_URI = "git://github.com/example/hello-go.git;branch=main;protocol=https;destsuffix=${GO_SRCURI_DESTSUFFIX}"
SRCREV = "0123456789abcdef0123456789abcdef01234567"

inherit go-mod go-mod-update-modules

require ${BPN}-go-mods.inc
require ${BPN}-licenses.inc

go-mod inherits the normal go class and adds support for module-aware builds. GO_IMPORT is still important: it tells the Go class the import path used for the source tree. GO_INSTALL says which package to build and install.

The destsuffix=${GO_SRCURI_DESTSUFFIX} part puts the fetched source where the Go class expects it. In practice, that means under a path like:

${UNPACKDIR}/${BP}/src/example.com/hello-go

You should not need to write your own do_compile() for the normal case. Let the Go classes set up GOOS, GOARCH, GOROOT, GOPATH, GOMODCACHE and the cross-compile flags.

Fetching Module Dependencies

The recipe above includes two generated files:

require ${BPN}-go-mods.inc
require ${BPN}-licenses.inc

These are maintained by go-mod-update-modules.

Run this when the upstream Go dependencies change:

bitbake hello-go -c update_modules

When creating a brand new recipe, those include files do not exist yet. I normally create empty placeholder files or temporarily leave the require lines out, run update_modules, then commit the generated files and the recipe together. Once the recipe is in normal use, I prefer require because I want the build to fail loudly if the dependency metadata is missing.

That task reads the module graph and writes:

hello-go-go-mods.inc
hello-go-licenses.inc

The hello-go-go-mods.inc file appends the module downloads to SRC_URI. It will contain entries with this shape:

SRC_URI += "\
    gomod://github.com/fatih/color;version=v1.18.0;sha256sum=<sha256> \
    gomod://github.com/mattn/go-colorable;version=<version>;sha256sum=<sha256> \
    gomod://github.com/mattn/go-isatty;version=<version>;sha256sum=<sha256> \
"

The exact list depends on the real go.mod and go.sum. The point is that the module downloads are now BitBake fetcher inputs, with checksums, so they can be mirrored, audited and fetched before compile starts.

The hello-go-licenses.inc file extends LICENSE and LIC_FILES_CHKSUM for the module dependencies it found. You still need to review the result. Generated license metadata is useful, but it is not a substitute for understanding what you are shipping.

Vendoring

There is another current class worth knowing about: go-vendor.

go-vendor is useful when you want a vendor-style build. It fetches module dependencies during the fetch/unpack part of the build and assembles a vendor directory for the module, rather than relying on the Go tool to download modules during compile.

The upstream Go command for creating the vendor manifest is:

go mod vendor

That creates:

vendor/
vendor/modules.txt

In a Yocto recipe using go-vendor, dependency source still has to be represented in SRC_URI. Wrynose provides a helper, go_src_uri(), for this:

inherit go-vendor

GO_IMPORT = "example.com/hello-go"
GO_INSTALL = "${GO_IMPORT}/cmd/hello-go"

SRC_URI = " \
    git://github.com/example/hello-go.git;branch=main;protocol=https;name=app;destsuffix=${GO_SRCURI_DESTSUFFIX} \
    ${@go_src_uri('github.com/fatih/color', 'v1.18.0')} \
    file://modules.txt \
"

SRCREV_FORMAT = "app_github.com.fatih.color"
SRCREV_app = "0123456789abcdef0123456789abcdef01234567"
SRCREV_github.com.fatih.color = "abcdef0123456789abcdef0123456789abcdef01"

That is deliberately only a small fragment. A real vendored recipe must list every module needed by vendor/modules.txt, including transitive modules, with fixed revisions. This is why I normally use the module include-file workflow first, unless there is a good reason to vendor.

Vendoring is useful when:

  • the project already maintains a vendor manifest
  • you need a build that mirrors the upstream vendor layout
  • the dependency source is easier to audit as a vendor tree
  • you are dealing with private modules or module proxy restrictions

It is not a reason to hide dependency management. The dependencies still need to be explicit, versioned and fetchable by BitBake.

What Not To Do

Older Go recipes often used:

export GO111MODULE = "off"

That was a way of forcing old GOPATH-style behaviour when module support became the Go default. It is not the modern answer for Wrynose.

For current Yocto projects, treat go.mod as part of the source contract. Use go-mod for module-aware builds, represent module downloads in BitBake metadata, and use go-vendor when a vendor-based workflow is genuinely useful.

The mental model is simple enough:

upstream Go development:
  go.mod -> go build may download modules

Yocto build:
  go.mod -> recipe metadata -> do_fetch gets all sources -> do_compile builds offline

That difference is the whole point. Yocto is not just compiling your Go program; it is recording exactly what went into the product.

Related posts

Browse all posts