Make final debugging terminal more reliable

[kienstra]

Background

Local CI has 3 terminals:

1. Main terminal: shows the job log as it runs
2. Debugging terminal: bash access to the job as it runs (but exits when the job finishes)
3. Final debugging terminal: bash access after the job exits

The final debugging terminal is needed because the debugging terminal exits when the job finishes.

But that terminal has been flaky, and sometimes it doesn't start a container at all.

It works by doing docker run on a committed container:

local-ci/src/utils/runJob.ts

Line 190 in d5a40f4

docker run -it --rm -v ${volume} $(docker images ${committedImageRepo} --format "{{.ID}} {{.Tag}}" | sort -k 2 -h | tail -n1 | awk '{print $1}')`

And the committed container is from doing docker commit of the job container:

local-ci/src/utils/commitContainer.ts

Line 24 in d5a40f4

committed_image=$(docker commit --pause=false $running_container ${imageRepo}:$(date +"%s"))

But sometimes there won't be an image for the final terminal to run.

[kienstra]

Removing the --rm flag in https://github.com/circleci-public/circleci-cli doesn't work.

diff --git a/local/local.go b/local/local.go
index 01fa87d..0212d9c 100644
--- a/local/local.go
+++ b/local/local.go
@@ -290,7 +290,7 @@ func writeStringToTempFile(data string) (string, error) {
 
 func generateDockerCommand(configPath, image, pwd string, arguments ...string) []string {
        const configPathInsideContainer = "/tmp/local_build_config.yml"
-       core := []string{"docker", "run", "--interactive", "--tty", "--rm",
+       core := []string{"docker", "run", "--interactive", "--tty",
                "--volume", "/var/run/docker.sock:/var/run/docker.sock",
                "--volume", fmt.Sprintf("%s:%s", configPath, configPathInsideContainer),
                "--volume", fmt.Sprintf("%s:%s", pwd, pwd),

It does prevent containers from being removed after exiting.

But this error appeared:

Error: 
Unexpected environment preparation error: error creating ephemeral network bridge: error creating ephemeral docker network: Error response from daemon: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

Step failed
Task failed