CI/Dockerfile: port 1.8 changes to 1.9

[pjonsson]

Cherry-pick the changes to CI/Dockerfile from 1.8 to 1.9. Cherry-picking commit 9579b4d gives a lot of conflicts, so make a separate small commit that removes the dash in docker-compose which are the necessary changes for this PR.

---

📚 Documentation preview 📚: https://datacube-ows--1049.org.readthedocs.build/en/1049/

[pjonsson]

Edit: first error was a merge-error from me, updated error and added the question about docker-compose.cleandb.yaml.

Edit 2: tests were failing because they were using an image that did not contain the changes in this PR. Added a --build to test.yml to get this through the CI. My thoughts about squashing vs amending/fixups and my question about docker-compose.cleandb.yaml are still relevant.

I don't know much about git, so I generally try to stay in my lane. I'm guessing it was a squash in the CI merge command into what is now commit 9579b4d. I don't think it's possible to avoid pain completely, but in general, I think it would be easier to take the pain and amend/fixup broken commits in pull requests and force-pushing so you get a "clean" PR/commit history, rather than adding commits to the PR and squashing everything when merging. With a reasonable commit hygiene, that strategy would reduce the size of merge conflicts, and that would make it easier to keep the two branches in sync.

Commit 1c22b2c changes the docker-compose.db.yaml file, should the same changes have been made to docker-compose.cleandb.yaml?

I don't see why this PR would make any change to these parts, but the failing CI job cannot find the datacube config:

+ datacube system init
Error: No datacube config found
Error: Process completed with exit code 137.

Do you by any chance happen to know what I might have messed up?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.27%. Comparing base (722bc6e) to head (705b91c).
Report is 9 commits behind head on develop-1.9.

Additional details and impacted files

@@               Coverage Diff               @@
##           develop-1.9    #1049      +/-   ##
===============================================
- Coverage        93.77%   93.27%   -0.50%     
===============================================
  Files               46       53       +7     
  Lines             6573     7038     +465     
===============================================
+ Hits              6164     6565     +401     
- Misses             409      473      +64     

see 8 files with indirect coverage changes

[SpacemanPaul]

I don't know much about git, so I generally try to stay in my lane.
in general, I think it would be easier to take the pain and amend/fixup broken commits in pull requests and force-pushing so you get a "clean" PR/commit history, rather than adding commits to the PR and squashing everything when merging.

You make a good point, but the reality with open source is the contributors rarely do take the pain, and I don't have the time to do it for them so my habit is to squash-merge rather than tangle master more than it already is, especially while I'm also working on develop-1.9.

But now I know that's how you prefer to work, I'll try to make sure I don't squash your PRs in future!

Thanks so much for all these PRs - your contributions are greatly appreciated.