Reduce image size

[ahans]

The current runner image was the best compromise of reasonable size and low effort for setting it up. However, it is still ~350 MB, which is a lot for what it does. It should be possible to bring it down to under 100 MB by using something like alpine as a base and building a custom GCC. Maybe we could even use something like tcc to get it even smaller. After all, we only need C for the tests, so tcc would most probably suffice.

Also, we add > 40 MB for Python which we only use to read the test output and turn it into the expected JSON format. Using some compiled language to produce a binary we should get away with much less. C++ would be quite easy for this, we could even keep the regular expressions we currently have in Python. This could happen in a two-stage image build, so only the final executable would end up in runner image.

@keiravillekode What do you think?

[keiravillekode]

This all sounds excellent. It is not something I have experience with.

[ahans]

Ok, I will see if I can come up with something clean.

[ahans]

I tried a ton of things, but now things slowed down again, so documenting here, my findings to not forget..

I tried out the following approaches:

• distroless base image with custom compiled TinyCC and binutils assembler plus custom compiled qemu-aarch64. This gives by far the smallest image and most things work (~80 MB incl. full Python). However, it looks as if TinyCC doesn't support all arm64 relocation modes so at least one example does not link with TinyCC reporting and unsupported relocation mode. We could probably solve this by not using TinyCC for linking, but GNU ld directly. However, that would require significant changes to the makefiles. Another solution could be a patch for TinyCC to support the missing relocation modes, but that's probably a major effort. Lastly, it's possible that when using TinyCC also as an assembler, those unsupported relocation modes would not be used in the first place. However, I expect that the TinyCC assembler would choke on the instructions that now lead to errors when trying to link.
• Using Alpine as a base and a custom GCC built is another approach. There's a ready-made package for building GCC with custom options against musl that gives small images as well and works. I was even able to fake a new user land so that existing make files work without having to touch anything. That gives relatively a relatively small image (~120 MB with Python), however I don't consider it exactly clean due to the fake-musl-for-glibc I have to do there.
• from what I know so far, I think the cleanest and best solution is a distroless base image (which uses binaries from Debian and 12, so it is gnu/glibc) with manually built GCC, binutils, and qemu-aarch64. Unfortunately, so far I was not able to come up with the right incantation for the GCC build that produces a binary that looks for all the headers and libraries in the right places, but that's the path I want to follow further. That would probably result in an image that has a similar size as the Alpine one.