Support custom environment at the OSArch level

[IainSteers]

We have a couple of cross-platform CGo projects that require custom CC and CXX specified per os-arch combination.

This is problematic because distgo currently only supports environment specification at the build level. Currently we're working around this by using multiple dist-plugin files and passing them to godel directly with the --config flag.

This isn't great because then the manifest and metadata files in the dist aren't entirely accurate, and it's hacky in CI.

As a solution, I'm envisaging something like:

build:
  environment:
    CGO_ENABLED: "1"
  main-pkg: ./main
  os-archs:
    - os: darwin
      arch: amd64
      environment:
        CC: /path/to/macos-cc
        CXX: /path/to/macos-cxx
    - os: windows
       arch: amd64
       environment:
         CC: /path/to/win64-cc
         CXX: /path/to/win64-cxx

[nmiyake]

Thanks for filing this! The configuration looks reasonable to me. I think it would be most natural for the os-arch build to use any globally specified environment values first, and then overlay any os-arch specific values on top of that (so if the same key is specified in both, then the os-arch level value will overwrite).

One possible concern I have is that I believe the os-arch struct is used pretty extensively throughout, and conceptually represents just the OS/arch combination. If we do proceed with this approach, then I think we should define a new struct that wraps the OS/arch struct and then use that struct in this section of config (and in the places in the codebase where it makes sense).

Also, a few questions about the CC and CXX values:

• What are the typical values for the CC and CXX path values? It seems like this would be pretty dependent on environment... Are the values in the CI build environment generally the same as the value in the local environment?
• Are the artifacts for different environments typically built on the same machine, or is it almost always the case that a different machine/image is used to build the different artifacts?
  • If it's the latter, then could it make sense for the values to be uniform, but just have symlinks for different platforms at the path?

Just want to make sure that the problem space is fully understood before we move forward on an implementation.

[IainSteers]

Hey Nick,

Thanks for the suggestions, agreed on your concerns re: the wide consumption of the OSArch struct.

In response to your questions:

• Variables are CI dependant. We could probably definitely make them the same via symlink trickery though. They're usually something like /usr/bin/mingw-w64-gcc or /usr/bin/osxcross-gcc
• All artifacts are built in the same linux container image environment as a CI step.

[nmiyake]

Thanks. So for point (2), that means that the Linux container image contains the osxcross-gcc binary and uses that for the darwin builds?

[IainSteers]

Yeah that's correct. The linux image has cross-compilation toolchains for windows, linux and darwin

[nmiyake]

Got it, that makes sense. Given all of this, I'm in sync with moving forward with the original proposal (as long as the environment information for os-arch is added in a scoped manner)