Add a dockerfile to build CAM-SIMA #222

K20shores · 2023-06-05T19:38:15Z

Adds a dockerfile to build an example case that uses CAM-SIMA
I based much of this work off of the ESCOMP-Containers repository, just as an FYI

This will allow the fine-grain testing of newly-added components to CAM-SIMA.

…the fork

K20shores · 2023-06-05T19:50:34Z

This is in draft because CAM-SIMA doesn't run successfully; I get a SIGILL. I am trying to debug why

K20shores · 2023-06-13T16:30:37Z

@nusbaume If you follow the instructions in the readme for building first esmf, and then cam sima, you should be able to start the container and run ./case.submit and see a run of cam-sima happen in docker.

nusbaume · 2023-07-13T15:46:51Z

@K20shores It looks like your new test-musica PR (#225) contains many of the same container files. Does that PR supersede this one (and so we can go-ahead and close this), or did you still want this to come in separately?

K20shores · 2023-07-13T16:43:10Z

@nusbaume I branched off of this branch to do the work in #225. I suppose that specific work in #225 really only tests a build of CAM-SIMA with Musica. If you don't care to have a base image that doesn't include Musica, we can abandon this PR.

nusbaume · 2023-07-13T17:19:20Z

I think since the vast majority of our eventual model configurations will want some type of chemistry, and because it would be good to test even seemingly independent physics changes with MICM linked in to make sure we aren't breaking that link, I would probably vote to only have one container (at least for now) that includes Musica.

@mattldawson @peverwhee @mwaxmonsky @boulderdaze Would any of you disagree with this thought process? Happy to be over-ruled here.

mattldawson · 2023-07-13T17:30:25Z

I think since the vast majority of our eventual model configurations will want some type of chemistry, and because it would be good to test even seemingly independent physics changes with MICM linked in to make sure we aren't breaking that link, I would probably vote to only have one container (at least for now) that includes Musica.

@mattldawson @peverwhee @mwaxmonsky @boulderdaze Would any of you disagree with this thought process? Happy to be over-ruled here.

Hi @nusbaume,

I have no strong feelings here, although if forced to choose, I would probably opt to be able to build CAM-SIMA images with and without MUSICA. Even if most of the time people will use MUSICA with CAM-SIMA, it could still be useful to be able to build and test them independently to ensure that they don't become too entangled. This could make it easier when someone wants to use a different chemistry (like GEOS-Chem). But, I'm fine with either option.

nusbaume · 2023-07-13T19:11:04Z

@mattldawson That's a good point, I hadn't thought about things like GEOS-Chem. I am fine to keep both containers then. I'll still plan on getting to the Test musica PR first as I imagine it is required for any future chemistry work, but will then put this PR as second on my to-do list, unless I hear otherwise. Thanks!

better language

nusbaume

Looks good, just have a couple Dockerfile questions and then a request to download the input data from a server instead of including it with the repo itself.

Dockerfile

README.md

nusbaume

Looks a lot better! I just have one last question/request.

nusbaume · 2023-07-28T17:35:01Z

Dockerfile

+RUN mkdir -p /home/cam_sima_user/cesm_data/inputdata/atm/cam/inic/homme/
+RUN cp /home/cam_sima_user/cami-mam3_0000-01_ne5np4_L30.140707.nc /home/cam_sima_user/cesm_data/inputdata/atm/cam/inic/homme/


Did you ever determine if you need this file? If not then it might make sense to not bother with trying to download it, and to remove these two commands.

I removed them and it ran. I pushed changes that removed this.

cacraigucar · 2023-07-31T15:35:18Z

docker/ftp_download.sh

@@ -0,0 +1,19 @@
+#!/bin/sh
+
+FTP_HOST="cesm-inputdata-lowres1.cgd.ucar.edu"


It this a good site to have the ftp data reside? Two concerns:

Typically the CGD ftp site has an expiration data available on it. I'm not sure if that policy is for this site as well.

With the threat of the removal of izumi in the future, I'm not sure it that will expand to wherever this site is located as well.

This is just a temporary solution until we start using snapshot files with the ne3 grid that have been processed to be as small as possible, which we will then bring in using the standard inputdata server methods.

The reason we haven't done that already is simply because there isn't an agreed-upon plan (yet) for where to put test data on the inputdata server, and because these files are larger than they need to be and so we didn't want to add them to a place where they couldn't be deleted but would stop being used in a few months.

@nusbaume or @briandobbins will better know how to answer this questions.

README.md

nusbaume

Looks great to me now. Thanks!

K20shores · 2023-07-31T16:27:30Z

@cacraigucar I have added some installation instructions for docker desktop to the readme. Do they seem comprehensive enough to you?

README.md

peverwhee

one small typo; otherwise looks good (though I did not try to build the docker image as Cheryl has fallen on that sword)

peverwhee · 2023-08-01T22:55:41Z

README.md

+cd docker
+docker build -f Dockerfile.esmf -t esmf .
+```
+2. Build the CAM-SIMA iamge


typo - image

mwaxmonsky · 2023-08-02T21:52:36Z

@K20shores Is there anyway to verify the case ran successfully in the container beyond that it generated log without error?

K20shores · 2023-08-03T03:10:23Z

@K20shores Is there anyway to verify the case ran successfully in the container beyond that it generated log without error?

not that I'm aware of. @nusbaume would know better

README.md

nusbaume · 2023-08-03T21:09:12Z

@K20shores Feel free to merge this PR whenever you're ready. Thanks!

again trying to make the branch play well with the main branch after …

2ad1f87

…the fork

K20shores requested review from cacraigucar, peverwhee and nusbaume June 5, 2023 19:38

adding comments

4acbd06

K20shores changed the title ~~again trying to make the branch play well with the main branch after …~~ Add a dockerfile to build CAM-SIMA Jun 5, 2023

K20shores added 8 commits June 5, 2023 15:39

correcting dockerfile...

fdb97fb

splitting esmf into its own image

61ece22

copying mam3 grid

c917459

adding jesse's updates

ae39850

adding jesse's changes

c571d4d

Merge branch 'docker' of https://github.com/ESCOMP/CAM-SIMA into docker

a59071c

Merge branch 'development' into docker

7fc89b0

it works

32421d3

K20shores marked this pull request as ready for review June 13, 2023 15:50

Update .dockerignore

d3f53d0

better language

nusbaume requested changes Jul 23, 2023

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

Dockerfile Outdated Show resolved Hide resolved

K20shores added 2 commits July 26, 2023 13:15

using ftp server

f918dde

removing file

65687bf

cacraigucar reviewed Jul 26, 2023

View reviewed changes

README.md Show resolved Hide resolved

downloading nc files over ftp in binary

679817b

nusbaume requested changes Jul 28, 2023

View reviewed changes

K20shores mentioned this pull request Jul 31, 2023

Test musica #225

Merged

removing not used file

9badced

cacraigucar requested changes Jul 31, 2023

View reviewed changes

nusbaume approved these changes Jul 31, 2023

View reviewed changes

adding docker installation instructions

07e07c2

cacraigucar reviewed Aug 1, 2023

View reviewed changes

README.md Show resolved Hide resolved

peverwhee approved these changes Aug 1, 2023

View reviewed changes

typo

493ec76

cacraigucar approved these changes Aug 3, 2023

View reviewed changes

README.md Show resolved Hide resolved

K20shores merged commit 774f71d into development Aug 3, 2023
12 checks passed

K20shores deleted the docker branch August 7, 2023 15:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a dockerfile to build CAM-SIMA #222

Add a dockerfile to build CAM-SIMA #222

K20shores commented Jun 5, 2023 •

edited

Loading

K20shores commented Jun 5, 2023

K20shores commented Jun 13, 2023

nusbaume commented Jul 13, 2023

K20shores commented Jul 13, 2023

nusbaume commented Jul 13, 2023

mattldawson commented Jul 13, 2023 •

edited

Loading

nusbaume commented Jul 13, 2023

nusbaume left a comment

nusbaume left a comment

nusbaume Jul 28, 2023

K20shores Jul 31, 2023

cacraigucar Jul 31, 2023

nusbaume Jul 31, 2023

K20shores Jul 31, 2023

nusbaume left a comment

K20shores commented Jul 31, 2023

peverwhee left a comment

peverwhee Aug 1, 2023

K20shores Aug 2, 2023

mwaxmonsky commented Aug 2, 2023

K20shores commented Aug 3, 2023

nusbaume commented Aug 3, 2023

		RUN mkdir -p /home/cam_sima_user/cesm_data/inputdata/atm/cam/inic/homme/
		RUN cp /home/cam_sima_user/cami-mam3_0000-01_ne5np4_L30.140707.nc /home/cam_sima_user/cesm_data/inputdata/atm/cam/inic/homme/

		@@ -0,0 +1,19 @@
		#!/bin/sh

		FTP_HOST="cesm-inputdata-lowres1.cgd.ucar.edu"

Add a dockerfile to build CAM-SIMA #222

Add a dockerfile to build CAM-SIMA #222

Conversation

K20shores commented Jun 5, 2023 • edited Loading

K20shores commented Jun 5, 2023

K20shores commented Jun 13, 2023

nusbaume commented Jul 13, 2023

K20shores commented Jul 13, 2023

nusbaume commented Jul 13, 2023

mattldawson commented Jul 13, 2023 • edited Loading

nusbaume commented Jul 13, 2023

nusbaume left a comment

Choose a reason for hiding this comment

nusbaume left a comment

Choose a reason for hiding this comment

nusbaume Jul 28, 2023

Choose a reason for hiding this comment

K20shores Jul 31, 2023

Choose a reason for hiding this comment

cacraigucar Jul 31, 2023

Choose a reason for hiding this comment

nusbaume Jul 31, 2023

Choose a reason for hiding this comment

K20shores Jul 31, 2023

Choose a reason for hiding this comment

nusbaume left a comment

Choose a reason for hiding this comment

K20shores commented Jul 31, 2023

peverwhee left a comment

Choose a reason for hiding this comment

peverwhee Aug 1, 2023

Choose a reason for hiding this comment

K20shores Aug 2, 2023

Choose a reason for hiding this comment

mwaxmonsky commented Aug 2, 2023

K20shores commented Aug 3, 2023

nusbaume commented Aug 3, 2023

K20shores commented Jun 5, 2023 •

edited

Loading

mattldawson commented Jul 13, 2023 •

edited

Loading