Articles

Speeding up Flashing eMMC for the STM32MP1

Improving the time to flash eMMC by replacing STM32CubeProgrammer with snagboot

embeddedLinuxPublished 03/09/20268 min read

STM32MP1STM32CubeProgrammerSnagbootU-BootDFUeMMC

Speeding up Flashing eMMC for the STM32MP1

I had an STM32MP157 board with a working flashing process based on STM32CubeProgrammer. It programmed the eMMC correctly, but having got used to the speed of flashing the microSD card with bmaptool and .wic files, I was getting frustrated with how long it was taking to flash the eMMC.

Snagboot looked like a good starting point. It supports recovery and flashing on STM32MP1, including booting a board from its ROM DFU mode.

What STM32CubeProgrammer was doing

The existing process used an ST FlashLayout TSV file describing the images and where they should be programmed.

For an STM32MP1 booting from eMMC, however, “flash the eMMC” does not mean writing one disk image to one block device. There are three areas I needed to program:

eMMC boot partition 1
eMMC boot partition 2
the main eMMC user area

The two boot partitions contain the first-stage boot firmware. The normal partition table, root filesystem and the rest of the system live in the user area.

There was another complication: when the processor starts in USB recovery mode, there isn’t yet enough running on the target to access the eMMC in the way I needed.

What I needed to do was:

STM32CubeProgrammer wraps most of that up behind its programming interface.

Starting from ROM DFU

The board was configured to start in the STM32MP1 USB recovery mode. On the host this appeared as a USB device:

0483:df11

Snagrecover supports the STM32MP1 recovery mechanism directly. My Yocto project build creates the required programmer files, so I could copy them from the deploy directory.

I could therefore start the temporary programmer with:

snagrecover \
    -s stm32mp15 \
    -F "{'tf-a': {'path': 'tf-a-programmer.stm32'}}" \
    -F "{'fip': {'path': 'fip-programmer.bin'}}"

This bootstraps enough firmware into the board for the next stage of programming.

DFU kept changing underneath me

One useful debugging tool during this work was simply:

dfu-util -l

At one stage, for example, I could see:

Found DFU: [0483:df11] ... alt=3, name="@PMIC/..."
Found DFU: [0483:df11] ... alt=2, name="@OTP/..."

The exact interfaces available depended on which stage the STM32 programmer was in. This became particularly important after sending the FlashLayout.

I initially treated the FlashLayout mostly as a description of what needed to be programmed. In practice it also an essential part of the STM32 programming protocol. Sending it causes the programmer to move on to the partition-programming phase and re-enumerate its DFU interfaces.

My flasher therefore couldn’t just send the FlashLayout and immediately assume the next DFU operation would work. It explicitly waits until the required partition interface appears:

def wait_for_partition_dfu():
    log("Waiting for partition DFU interface...")

    deadline = time.monotonic() + 10

    while time.monotonic() < deadline:
        SnagbootUSBContext.hard_rescan()

        usb_addr = find_usb_path(DFU_VID, DFU_PID)

        if usb_addr is None:
            time.sleep(0.1)
            continue

        dev = get_usb(
            usb_addr,
            error_on_fail=False,
        )

        if dev is None:
            time.sleep(0.1)
            continue

        if has_altsetting(dev, 10):
            log("Partition DFU interface ready.")
            return dev

        usb.util.dispose_resources(dev)
        time.sleep(0.1)

    raise FlashError("Timed out waiting for partition DFU interface")

This is one of those details that is easy to miss when using a vendor programming tool. From the outside it looks like one USB programming session. Underneath, the target is changing state and exposing different interfaces as it progresses.

Programming the eMMC boot partitions

After processing the FlashLayout, the programmer was ready for the eMMC boot partitions.

Boot partition 1 could be written using its DFU alternate setting:

download_file_to_device(
    partition_dev,
    0,
    tf_a_emmc,
)

Boot partition 2 required another STM32 programmer phase.

The programmer exposes a virtual DFU interface through which a five-byte command can be sent. The first byte selects the phase and the remaining four bytes contain the address or offset.

For my case the offset was zero:

payload = bytes(
    [
        phase,
        0x00,
        0x00,
        0x00,
        0x00,
    ]
)

I could then select phase 0x05 before programming boot partition 2:

select_phase(
    0x05,
    virtual_alt=10,
)

download_file(
    1,
    tf_a_emmc,
)

At this point I had reproduced the part of the STM32CubeProgrammer process responsible for installing the firmware into both eMMC boot areas.

DFU is slow…

The installer image for this system is generated by Yocto as a compressed WIC image together with its bmap:

installer-image.rootfs.wic.xz
installer-image.rootfs.wic.bmap

I could have continued trying to reproduce more of STM32CubeProgrammer’s programming behaviour through DFU. But by this stage there was already a perfectly capable piece of software running on the board: U-Boot.

Rather than invent another way to transfer a large disk image through the STM32 programmer protocol, I could get U-Boot to expose the eMMC directly to the host using the USB Mass Storage:

ums 0 mmc 1

Reusing the U-Boot that was already running

The temporary programmer firmware includes U-Boot, so after finishing with the STM32 programmer I could send a DFU detach rather than rebooting the board:

def detach_stm32_programmer(virtual_alt: int):
    dev = find_dfu_device()

    try:
        dfu_detach(dev, virtual_alt)
    finally:
        usb.util.dispose_resources(dev)

That shuts down the DFU gadget while leaving me with the existing U-Boot instance.

I could then use the board’s serial console to issue the UMS command:

console = serial.Serial(
    serial_device,
    baudrate=115200,
    timeout=0.2,
    write_timeout=2,
)

time.sleep(0.5)

console.reset_input_buffer()
console.reset_output_buffer()

console.write(b"\r")
console.flush()

time.sleep(0.1)

console.write(b"ums 0 mmc 1\r")
console.flush()

There was some investigation involved here because sending a command to a U-Boot serial console immediately after shutting down another USB gadget isn’t quite the same as typing it into an idle prompt.

The final sequence includes the short delay, sends a carriage return first and then sends the UMS command. Once that succeeds, the board disappears as the STM32 DFU device and reappears as a USB device:

0483:5720

More importantly, Linux now sees the eMMC as an ordinary SCSI block device such as:

/dev/sdb

This meant I could now write directly to the eMMC.

Finding the right block device matters

Automatically writing a WIC image to /dev/sdX deserves a certain amount of paranoia. Choosing the wrong disk would make for a rather different Tales from the Trenches article.

I didn’t want the script to assume that the newly appearing /dev/sdX was necessarily the target.

Instead, I walked the Linux sysfs hierarchy for each candidate block device until it finds the parent USB device. It then checks that device’s USB VID and PID:

if vid.lower() != f"{UMS_VID:04x}":
    continue

if pid.lower() != f"{UMS_PID:04x}":
    continue

I also captured the STM32 USB serial number at the start of the process and compare it with the serial number of the UMS device. That means the script can follow the same physical board as it changes from DFU into USB Mass Storage.

There is another sanity check before anything is written: the size of the discovered disk must fall within the expected range for the eMMC fitted to this hardware.

None of these checks make the flashing itself faster. They just make automating the flashing considerably less exciting.

Writing the WIC with bmap

Once U-Boot exposes the eMMC as a normal block device, the actual user-area programming becomes straightforward.

I used Snagflash’s bmap_copy():

with open(device, "rb+", buffering=0) as destination:

    bmap_copy(
        str(image),
        destination,
        None,
    )

    destination.flush()
    os.fsync(destination.fileno())

Again, I was able to make use of the artifacts from the Yocto build as I had configured it to create .wic.xz files.

installer-image.rootfs.wic.xz
installer-image.rootfs.wic.bmap

The WIC image describes the complete eMMC user-area layout I want, while the bmap describes which blocks actually contain data. There is therefore no reason for my flashing script to understand the individual Linux partitions within the WIC image.

At this stage it is just writing the disk image produced by the build system. It also means the large user-area image doesn’t need to go through the STM32 programmer’s skow DFU partition protocol.

The complete flashing process became:

Did it make flashing any faster?

Using time with both methods provided the following results.

STM32CubeProgrammer:

real    3m26.188s
user    0m4.264s
sys     0m5.277s

The new flasher:

real    1m58.553s
user    0m6.557s
sys     0m0.690s

That takes the elapsed flashing time from 206.188 seconds to 118.553 seconds.

The difference is 87.635 seconds per board, or about a 42.5% reduction in elapsed time.

While I wouldn’t treat two time measurements as a detailed performance benchmark as USB topology, image contents, eMMC performance and host behaviour can all affect the result, ot does give a good indication of the improvement.

For the actual workflow I cared about, though, dropping from around three and a half minutes to just under two minutes is useful. This becomes more significant when the same operation is being repeated during development or across multiple units in manufacturing.