By: Lofty Solution Architect Christopher Hobbs

DISCLAIMER: This post is the result of work done with small sample sets and time constraints. It's merely the entrypoint for further tinkering rather than a presentation of complete ideas.

Here at Lofty, we manage k8s with Rancher and we often use Alpine images for high-speed/low-drag containers. When I joined Lofty a few short weeks ago, I learned that SUSE had purchased Rancher. In light of that and the (then) upcoming SUSECON Digital '21 event, I wanted to explore this partnership a little deeper.

I spent a lot of time with the openSUSE and SLES/D communies back in their Novell days and I was curious what they might bring to the containerized world. I took a lab day to investigate how openSUSE/SLE container images might stack up to Alpine images. This post highlights the results of that quick look.

Target Metrics

The contenders for these tests were the latest (as of 2021-05-14) opensuse/tumbleweed image from the OBS and the latest alpine image from Docker Hub.

I wanted to take a look at the overall weight of each distro in terms of package and executable count. As part of this, I hoped to explore the potential attack surface area and possible CVE comparisons between the two. I managed to enumerate the former while I ran out of time for the latter. The best measure I was able to come up with for vulnerability analysis given my time constraints was "Both images have CVEs just like any other image but openSUSE (Tumbleweed) seems a little more up to date at the time of testing."

Following weight and surface area, I hoped to put a load on a few containers. I initially started out with K6 for this test but settled on stress-ng in order to make things a little more simple. Both sets of containers were placed under reasonable loads and monitored for a short while. One minute load averages were captured for this metric.

Finally I hoped to test the network throughput of both image types. They're using practically the same network stack but I wanted to verify that they would be (nearly) equally performant given that most of what we build requires solid network response times. I used iperf3 for these measurements.

Packages and Executables

Looking at the total number of packages and executables in each image yields the following results:

• Apline executable count: 3,503
• Alpine package count: 14
• Alpine image size: 5MB
• openSUSE executable count: 5,012
• openSUSE package count: 95
• openSUSE image size: 82MB

We can see that openSUSE is obviously much larger than Alpine and this is exactly why we often use Alpine in our deployments. Given that the openSUSE image is almost 16 times the size of the Alpine image, it makes sense to fret over this sort of thing but I hold the position that storage is cheap and we're still in the sub 100MB range. Both are relatively compact images for what they do.

Stress Testing

I ran stress-ng for 20 seconds over three iterations on both sets of containers to collect one minute load averages. Each image ran under tests with 4 dedicated CPU cores running 4 sqrt() workers, 2 sync() workers, and 2 malloc()/free() workers each. Each worker was allocated 256MB of memory. We found that Alpine's one minute idle load average was 1.66 with an average of 3.13 while under stress. openSUSE clocked in idling at 0.66 with 2.99 under stress.

I'm still not quite sure why Alpine had a higher load average when idle and would like to do more investigations to see what might have been running in the background. Both images held up well under load, though openSUSE appeared to be slightly more performant. Future tests might include some concurrency or maybe disk operations while we're at it.

Throughput Testing

Running throughput tests with iperf3 yielded about what I would expect to see: virtually no difference in network performance between the images. I took the averages of 10 separate iperf3 runs with about 100 packets sent on each run for this test. Both sets of tests were done with no other network traffic on any of the test machines. Alpine came through with about 61.3 Mbits/sec while openSUSE gave us 63.3 Mbits/sec.

Both throughput graphs were nearly identical when plotted per packet. This was also expected. It appears that both distributions are just fine when considering (standard) network traffic.

Other Considerations

In an ideal situation, we would have a single image for each container. This would make a lot of things easy for us by establishing a known baseline. However, it's best to use the right tool for the right job and sometimes you need something different.

Alpine is based on BusyBox and Musl and while that's worked for a lot of our containers, it has presented some issues. Some of our work requires the GNU toolchain and utilities. Musl does provide a compatibility layer but it occasionally falls short and the option is to either try to add the necessary tooling to Alpine or find another image. Finding a new image is the obvious path of least resistance.

Alpine's intent is to be quite spartan. This lends itself well to performance and reduced surface area but if you ever need to do any investigative work in a container, you'll be left without any useful tools. While working directly in containers is generally discouraged, it happens often during development or forensic operations. openSUSE comes with a lot of batteries included (along with a few extra gadgets like AppArmor).

Community matters as well. It seems that SUSE has a larger and more tightly knit community. One potential downside is that SUSE is also a product and with that comes all the trappings of proprietary and commercial things. Its community is possibly inflated by employees and marketing. However, it has a greater array of support options as a result.

Finally, Alpine sports wider architecture support than openSUSE. While we are currently just using x86 variants, if we ever wanted to run containers on something exotic we may want to lean into Alpine. Exotic may also mean small and Alpine's tiny footprint would be a welcome feature in such a case.

Conclusion

My general conclusion is a non-conclusion...

Alpine is probably fine as it is but it could have some dark corners to watch given its foundation of BusyBox and Musl. It's far smaller than SUSE so that's a bonus if you're really concerned with disk space. Lightweight is a positive thing but it can create friction if you ever need to work directly in the container itself.

openSUSE is also probably fine as it is. It seems like tumbleweed receives updates more quickly than Alpine but it's also substantially larger. It seems to handle system stress better than Alpine (at least given my testing) and that might be related to the well polished default utilities. openSUSE comes with plenty of extra tooling that might help during development and production runs.

So which base image should you use? The one that works for you. I believe we will continue to use Alpine where we have it and consider openSUSE for future projects. We will definitely watch the SUSE world for future integrations with Rancher, k8s, Docker, etc. With exciting things like licensed SLE images on common cloud providers, SLE BCI, and various Rancher features discussed at SUSECON Digital '21, the future looks bright for this distro.

‍