RFC: use Docker credentials helper for Docker Hub login

[jamestelfer]

Is your feature request related to a problem? Please describe.

The built-in support for Docker Hub login relies on the registry user and password being available in the environment. When these are present, the default pre-command hook uses these variables to perform a docker login.

This solution works well: it's straightforward, and it supports retries on network failure.

If logging in on every step is not desirable, it is possible to use the login plugin directly, or use per-pipeline settings for the environment variables.

Why would logging in matter? Many steps don't use DockerHub images, but login takes ~1s per job. This doesn't seem like much, but it's one of the papercuts that lead to slower builds overall.

So, I'd like to avoid both logging in on every job and configuring the login for every pipeline.

Describe the solution you'd like

Use a Docker credential helper that retrieves login credentials from the environment, and enable it for the default registry.

In order to solve this for myself, I created https://github.com/jamestelfer/docker-credential-env

Then, in the hook, we configure the plugin using the credentials for the Docker registry and the Docker registry URL.

Describe alternatives you've considered

It's not 100% necessary to use the binary for this -- it's possible to implement a helper in Shell. Going the binary route allowed me to pull in the base helper library that Docker provides.

After going to the trouble of doing this I'm concerned that I've overlooked a more trivial solution, but hopefully the Internet will tell me and I'll learn 🤷‍♂️

Additional context

On reliability

The Docker Login plugin supports retry, and the proposed helper strategy does not.

On the face of it, that makes this solution less reliable. However, since the helper means that the credentials are only used to login when necessary, instead of occurring for each build, an issue connecting to Docker Hub only affects builds that use it. This decreases the overall blast radius of a login issue to only those builds that directly use Docker Hub.

In my circumstance, with a wide variety of workloads being run, failing the subset of builds that reference Docker Hub when there are issues is the way to go.

Previous to this, the solution used by my company was to fail silently if the login failed (not retrying) and let the chips fall as they may when it came to rate limits. Introducing the helper meant that we're more quickly vulnerable to Docker Hub login issues, but getting rid of the speed bump on most steps was worth it.