I've been using Alpine Linux as my primary container distribution for many years. I was originally attracted to it due to its simplicity and small size, and because certain architectural decisions felt like the right choices when it came to container suitability.

Alpine is still an excellent container distribution (when you can't use scratch), but as always, there are trade-offs. Below I'll discuss two aspects you may not think matter much (and often they don't), but can still surprise you down the line.

#BusyBox

I really like BusyBox.

Back at university, my flatmates and I had fun building our own routers and embedded devices, and BusyBox was vital for providing usable coreutils in highly resource-constrained environments.¹

BusyBox is known as The Swiss Army Knife of Embedded Linux. It's a single binary that contains implementations of over 300 common Unix utilities.

To use it, you can either invoke busybox <name> <args>... or create a symlink to the busybox binary with the name of the tool. BusyBox will look at argv[0] and decide which tool to invoke.

The About page describes the trade-off clearly (emphasis mine):

The utilities in BusyBox generally have fewer options than their full-featured GNU cousins; however, the options that are included provide the expected functionality and behave very much like their GNU counterparts.

It's not that surprising that BusyBox doesn't achieve full compatibility with the equivalent GNU tools, as that would be a huge undertaking. It's not BusyBox's intention to be one-for-one compatible with those tools, but it's close enough. This is acceptable when it's a human typing commands, because they can adjust for the differences.

But one day you'll build a container and discover that something inside it assumes GNU coreutils is available. It'll shell out to du with some obscure flag that doesn't exist in BusyBox, fail to handle the error, and... panic! at the disco.

#musl

I also really like musl, for similar reasons. If you know what glibc is, then you can imagine musl as a cleaner alternative libc implementation, unencumbered by decades of accumulated complexity, workarounds, and compatibility layers.

This all sounds great, so what's the downside?

One thing glibc has going for it is performance. glibc's code is complex, but that's partly due to all the optimisation work that's been done over its lifetime. musl's chosen path is simplicity over performance.

A little performance loss is fine if the application is not bottlenecked by libc, but when it is, musl is not ideal.

So what did I use instead?

#Will the real slim Debian please stand up?

Not all of my containers were affected by the issues above, but in the end I chose consistency over fragmentation, and just migrated everything to Debian slim images.

Debian may as well be considered the happy path: most upstream software assumes glibc, and scripts and dependencies are more likely to just work.

It's not that bad:

alpine:latest           3cb067eab609    8.45MB
debian:bookworm-slim    f54f5c8e2e12    74.8MB

I'll admit that optimising away the difference between 74MB and 8MB is not really necessary today, but it sure felt nice while it lasted.