Cross-compiling for a small device: the basics

Small devices are too slow to build on, so you build on a PC and run on the target. That is cross-compilation, and most problems come from a mismatch between the two sides.

What you must match

CPU architecture — for example x86-64 to AArch64 or ARMv7.
libc — a binary linked against glibc will not run on a musl system, and the reverse. This is the classic surprise on Alpine.
ABI details — for 32-bit ARM, the floating-point convention (hard-float vs soft-float) must agree.
Library versions — the target must have the shared libraries your program was linked against, at compatible versions.

Get a toolchain that matches the target

Let your build system produce it. Buildroot and Yocto build a toolchain and matching SDK for your exact image, which avoids mismatches.
Or use a prebuilt cross-compiler for your target and a sysroot copied from the device or image, so headers and libraries are the right ones.

The easy path: static binaries

Static linking bundles libraries into the executable, so libc and library versions on the device stop mattering. Go and Rust make cross-compiling and producing self-contained binaries straightforward, which is why they are popular for small devices. The cost is larger files and having to rebuild to pick up library security fixes.

Test before you deploy

Run the binary under QEMU user or system emulation to catch architecture and ABI errors on your PC.
Check what a binary expects with tools like file and readelf (architecture, interpreter, needed libraries).
Finally, test on the real hardware; emulation does not reproduce timing or peripherals.

Common failure messages

“No such file or directory” when the file plainly exists usually means the dynamic loader the binary asks for is missing — a libc mismatch.
“Exec format error” means the wrong architecture.
Missing .so errors mean a shared library is absent or too old on the target.

Keep the toolchain and build environment versioned so a build you did today can be reproduced later.