-
Notifications
You must be signed in to change notification settings - Fork 313
Meetings notes 2023 Software
- Assign timekeeper
- Jan 5th, 2023 -- First meeting of the new year
- Jan 12th, 2023 -- What needs to be done by the LMWG meeting? Winter Qtr
- Jan 26th, 2023 -- Last meeting of January
- Feb 9th, 2023 -- What do we need to adjust based on the LMWG meeting?
- Feb 23rd, 2023 -- Last meeting of February
- Mar 30th, 2023 -- Last meeting of March
- Apr 13th, 2023 -- Spring Qtr
- Apr 20th, 2023 -- SEA Conference cancelled
- Apr 27th, 2023 -- what did we learn from the SEA conference? Last meeting of April
- May 4th, 2023 -- Star Wars Day, More on SEA conference
- May 18th, 2023 -- What do we need ready for the CESM workshop?
- May 25th, 2023 -- Last meeting of May
- Jun 15th, 2023 -- cancelled, CESM Workshop
- Jun 22nd, 2023 -- What do we need to adjust based on the CESM workshop?
- Jun 29th, 2023 -- Last meeting of June
- Jul 13th, 2023 -- Summer Qtr
- Jul 27th, 2023 -- Last meeting of July
- Aug 31st, 2023 -- Last meeting of August
- Sep 28th, 2023 -- Last meeting of September
- Oct 5th, 2023 -- First meeting of FY2024
- Oct 12th, 2023 -- Fall Qtr
- Oct 26th, 2023 -- Last meeting of October
- Nov 23rd, 2023 -- Cancelled Thanksgiving
- Nov 20th, 2023 -- Last meeting of November
- Dec 14th, 2023 -- AGU
- Dec 21st, 2023 -- Last meeting of December
- Dec 28th, 2023 -- Cancelled Christmas
- Jan 4th, 2023 -- First meeting of the new year
- Jan 11th, 2023 -- What needs to be done by the LMWG meeting?
See Google Doc: CTSM Software Engineering Meetings 2023
- Timekeeper: Bill
- Ryan's PR with BGC call sequence
- Still working on #1959 (update to use BGC call sequence)
- FATES refactoring - still B4B, working on unit testing. For later - how should these kinds of non pass/fail tests get called?
- sub-question: FATES parameter file read-in subroutine is in the CLM codebase. This seems odd to me. Feels like FATES should have a parameter read-in subroutine that CLM calls. Ultimately, just makes unit-testing a bit more annoying the way it's set up now.
- Sam R. -- sample Google agenda. Lead us through how this works
- Keith: What's the best way to make a domain file for an atmospheric forcing dataset?
- Bill: run_sys_tests verbosity (https://github.com/ESCOMP/CTSM/issues/1508) (5 minutes)
- List for new people of things we need:
- Login to cheyenne/izumi
- github handle
- Add github to ESCOMP and CTSM groups
- Add to github externals they will work on
- Add to cseg and cgdcsm on cheyenne/izumi
- How to use rimport and standards for datafiles in inputdata
- svn password for inputdata
- Bill: see https://docs.google.com/document/d/1zAOcFWrXCDJoubUXrpqLGH_9qPDOpic7zV88fkGGK9U/edit
- Ryan's BGC call sequence PR
Keith: a user on the forum has their own atmospheric forcing dataset; how can they create a domain file?
If you use mknoocn, it will set the land mask to 1 everywhere. Erik thinks that may be what's done for current atm forcing datasets, so that makes sense as long as the dataset defines data everywhere. So can use that process together with other pieces of the process when creating lnd domain files.
- Timekeeper: Adrianna
- Restart memory reduction updates (FATES#991), no answer changes
- Still need reviewers for #1959?
- FATES refactoring - all B4B so far, figured out how to add netcdf to CMake!
- Use an app for timing? So we can see how well and what we follow?
- Agenda in google doc -- so can co write? This wiki could point to a different doc for each meeting or to meeting notes for the entire year. The google doc could be open for editing from the group that normally attends and global read-only for anyone with the link.
- Next steps for SNICAR updates PR #1861
- FATES use the normal BGC call sequence #1959
Would it be good to use an app for timing (visible to everyone)?
- Could be worth trying
- Sam: it feels like the most important thing is to see how much time we should allot to each section – e.g., what tends to take longer / shorter than the allotted time so we can make better time allotments moving forward.
- Sam's sense is that FATES updates could often use more time
- Will's sense is that short-term issues often take longer than allotted
Short-term vs long-term:
- Idea was that short-term things need input this week; long-term could be deferred until there's time in a meeting to cover it.
Sam suggests: Throughout the week, people can add items they want to discuss; indicate how long they think it will take and the priority; then it can be put in the appropriate section.
Using a google doc for meeting notes:
- Sam suggests having a google drive folder with a separate document for each week's meeting.
- Advantages would be that everyone can view and edit this at once
- A disadvantage could be searchability
- Pros and cons to having a separate document per week or one per year (separate documents for each week can make it easier to find the week when we discussed some particular thing based on a search for multiple words; one document per year can make it easier to quickly scroll / search through)
https://github.com/ESCOMP/CTSM/pull/1959
Ryan feels like the main question points on this have already been addressed by email, etc.
High-level: There are things that CTSM does in terms of tracking C & N that, up until now, FATES has not participated in. Instead, there were conditionals leading to a subset of things being done with FATES. This PR changes it so that previous conditionals on use_cn now check use_cn or use_fates_bgc.
Movement of summary variables (like totc_col) into the soil biogeochemistry type: This was needed because the type that initially contained these variables isn't allocated when running with FATES. We had discussed the possibility of creating a type just to have these summary variables, but Ryan's sense at the time was that people were okay with having these in the soil type.
Regarding CNVegetationFacade: Previously, FATES wasn't using this. Now FATES is using at least parts of it.
- Bill: the initial vision for this was that FATES might have its own version of this module, in an object-oriented sense. We had a bit more discussion of this as a longer-term thing; not clear if that would make sense or not.
- Ryan points out that this is entangled with CNDriver calls... this would need to be disentangled if we wanted to go the object-oriented route.
Summary: Both C & N code is in place, though N won't do much yet. Harvesting code is also written to accommodate FATES harvesting fluxes and run harvesting code - specifically referring to the product pools; currently FATES is passing 0 fluxes, but now the hooks are in place to allow non-zero values.
- Timekeeper: Will
- FATES tag and FUNITCTSM test working
- Split associate statement to avoid NAG compiler line continuation problem (FATES #1009)
- Migrate FATES to use BGC call sequence #1959
- The Great Refactoring
- Sam: Should I attend the SEA Improving Scientific Software conference?
- Keith: Do we need a project board for CESM3 Development simulations? Larger discussion, we'll decide on a subgroup to start, and then present to both CTSM software and CTSM science.
- Timekeeper: Sam R.
- Bill's tag with a difficult externals update is in, Dev_120!
- Dust work is close, will be meeting next week with CAM-Chem team.
- mizuRoute is close to being able to come in.
- Who is going to do the FATES updates with Jackie out of the picture? Likely Adrianna, but Greg/Ryan can fill in.
- Adrianna also planning to play (even more of) a leadership role CTSM-FATES development (both science and SE).
- Little to report out, lots of travel and absences on the FATES team.
- Soil moisture initialization changes with FATES are close to coming in for CTSM #1962.
- Erik: To send out website re. GitHub ssh changes.
- Erik: Talk about how to signal moving on for time-keeping in our meetings. How do we signal in a way that is comfortable to everyone? How do we make sure we aren't cutting people off? Or being inequitable in checking time?
- Rotate responsibility around the group (similar to note taking), We'll add this to the agenda, rotating alphabetically.
- Visual and auditory cue are helpful (although maybe too formal for this situation.
- Adding a countdown clock to desktop so everyone sees it.
- Raising hand could be a good way to do this?
- Sam: Can we designate an agenda item to space for "Here's a question/issue I've run into; can someone help me with it later?" (I put mine in "Short term issues/decisions" but it doesn't feel right there.)
- Add heading for Questions/issues to raise (similar to the standup's meeting).
- Erik: Next week at 10 is an SEA general interest meeting, can we run a short meeting next week?
- Yes.
- Sam: How to set up a custom single-point run with
release-clm5.0?- Keith posted this, which Sam said he tried, but ran into issues.
- Conda environment issues: #1974. Do we need to develop ways to prevent this, or force checks in ctsm_pylib?
- Negin had some suggestions on this issue thread.
- Keith and Erik are going to meet on this.
- FATES soil moisture will be coming in next,
- Regional subsetting to follow (Erik said it's close)
- Quick updates on coupled model simulations with CPL_HIST, #1951
- Resolving issues related to solar zenith angle.
- Air density being written over by CTSM (not cpl_hist), Keith has a hack.
- Keith has a number of code mods that need to be brought into main to ctsm, cmeps, etc. Keith will work with Bill regarding timing of when this comes in for spinup and running latest CESM tests.
- Not, a win, but Jackie is moving on to another great opportunity for her. We will miss her though and her contributions.
- Adrianna helped me get to the next step
- Great candidates for the next CGD director.
- We are 99% sure we figured out the mapping issue with mizuRoute
-
initialization problems - nocomp with large trees (FATES PR#995, more water (PR#1962)
- needs more discussion on CTSM side?
-
FATES to use BGC call sequence PR#1959
- in testing, needs reviewers
- graph and nodes for CTSM refactoring (Adrianna & Erik)
- Sam's next steps for Olga's FruitTree PR (#1966)
Small diffs in TOTN and TOTC. Feeling is: as long as this stays near roundoff (verifiable with summarize_cprnc_diffs), this is fine, since this is expected due to a change in order of operations.
Feeling is: for now, we can let this stay as is, with different initialization for FATES. We'll open a CTSM Discussion to decide what we want to do long-term.
- Actually, let's use an issue to ensure that this doesn't get forgotten, and because we will likely want some code changes to emerge from this.
Greg will bring this home.
Context: Gordon has been talking about bringing in the multi-layer canopy, but it requires some refactoring: currently, fluxes are calculated in different places, etc. In addition, Adrianna has been thinking about reworking things like albedo calculations to have them shared between CTSM and FATES.
This has led Adrianna to look into possible solutions for generating graphs showing relationships between subroutine calls as well as variables: how they're set, where they're used, etc. Is there some way to use the abstract syntax tree for this?
Some ideas:
- Things built in to editor (LSP - e.g., VSCode... but not sure if it does more than just a call tree, and maybe not even that for Fortran)
- https://graphlytic.biz/blog/code-refactoring-using-graphs
- Doxygen
- Something that was created by ORNL
- gprof
Adrianna will look into this more.
Ryan wonders if there would be value in creating a count of warnings in each PR so we can keep track of whether this is going up or down.
Ideally we'd clean them all up, but that would be a big project.
Erik: used to use FLINT for this.
-- Dave's back!
- 994 (bareground establishment): Increasing soil moisture and start with larger trees fixes initial mortality of trees. Charlie to open issue on CTSM. Here on CLM side (Ryan will do this update) https://github.com/ESCOMP/CTSM/blob/master/src/biogeophys/WaterStateType.F90#L351-L355. Check w/ Sean Swenson on implications of initialized higher soil moist.
- CTSM 1959 (nutrient coupling): use native calls in CTSM More heterotrophic resp leads to big soil C losses at BCI tropics site. New BGC sequence makes sense - tropics do not have much soil C. Needs more attention and review.
- 931 (Atkin respiration model): Implemented variable name and C4 updates. In testing.
- 958 Drought Phenology (Marcos) looks good. being integrated. 984 Acclimation (Charlie, Ryan) add after 931 Atkin.
- Calibration: Demographic benchmarking (Jessie): having hard time getting FATES to run w/ Trendy data. Reading a different date than file name.
- Erik: ctsm5.1.dev119 -- Peter is reviewing the code to make sure it makes sense with the changes that have happened. I highly support doing this, but think we should do the tag as is now. If Peter finds issues we can still make those changes on the ctsm5.2 branch.
- Erik: Sent a meeting invite for Friday to talk about github actions to Greg and Sam R.
- Erik: Bringing up issue for Bill that Naoki is using OCGIS for creating mesh files for mizuRoute. Since Ben is long gone this is a real problem for us long term.
- Ryan/Will: Hardcode nitrif_denitrif flags to .true.? This is problematic for a smallville test. Was an issue for FATES, but that's been changed.
- xxx
- xxx
To get more reasonable results with FATES, it seems we need higher initial soil moisture. The current implementation involves having different initial values for FATES vs. non-FATES.
Some of us feel that we should try to avoid having arbitrary differences like this: it can throw things off for users who – e.g. – are doing comparison studies of FATES vs non-FATES. A solution could be to increase the initial soil moisture for non-FATES as well as FATES.
Sam: if it isn't possible to use the same initial soil moisture for non-FATES, one possibility would be to require an explicit flag to turn on this difference.
- Erik: or have a namelist-settable initial soil moisture value.
- Bill: feels it would be better to just change things consistently for non-FATES and FATES... then just consider one of these namelist options if that causes problems.
- This will probably require a run to ensure that this doesn't break things.
Next steps: Discuss this in a more science-focused meeting so we have the right people in the room.
Plan is to bring this into master with some small test cases. Then with the 5.2 integration it will be more heavily exercised.
Will plan to bring this in as is, then make any changes needed on the 5.2 branch or directly on master.
Agreement that this makes sense. Probably wait until Monday-ish and then move ahead.
Erik: Bringing up issue for Bill that Naoki is using OCGIS for creating mesh files for mizuRoute. Since Ben is long gone this is a real problem for us long term.
There is currently a problem with how multi-polygons is done; talking with Bob about this. It seems like OCGIS may have problems with how it creates multi-polygons from the shape files.
mizuRoute starts with a shape file and converts it to a mesh; OCGIS does this.
Bill: the unfortunate reality is that the ESMF team has lost its python developers, so it's hard to promise any significant support for this package. There is some hope that this could change long-term – once we get an ESMF core team manager – but for now things are uncertain.
Sam: can the GDAL tools handle this? Erik will point this out to Naoki.
This is no longer an issue for FATES.
This is still set to false for a smallville test... but it looks like it may have been removed... or if it's still there, it might have only been done so we'd have a test of this configuration... so can just change / remove the test.
-- Our testing caught a change in answers in a diagnostic field in the GULU branch. By the way the GULU branch is a PR that was outstanding for 5 years.
- 931 (Atkin respiration model):(Charlie, Ryan) in testing. update to C4 to C3 value (same as CLM5). 986 (fates-sp issue) and 987 (SP memory): (Ryan) fixing overallocation to cohort array. in testing with large number grid cells. 988 (canopy spread issues): discussion to update this to data driven (Tallo, BAAD, TRY)
- Calibration: Adrianna and Jessie working together to confirm common configuration. Adrianna - tackling albedo bias.
- Will: CESM3 planning
- https://github.com/ESCOMP/CTSM/discussions/1951
- Erik: Have some actions for automatic python testing working now. I can show this later if people like.
- xxx
- xxx
- xxx
- xxx
-- Sam L. was able to get the GULU branch updated to the latest and should be next tag! -- CGD Culture survey general report was released! Get back to us on questions/concerns... -- CGD "On the Spot" recognition https://docs.google.com/forms/d/e/1FAIpQLSdyb485HYb_fmhpnwrWMfddI_OTxOt10AngFR7kmd157aGBnA/viewform
- C balance checking at the column-scale on the FATES side? (grid-scale on the CTSM side)
- We want to have the main part of this meeting an hour, and then a half hour for a given focused topic.
- Will and Erik will review the agenda 15 minutes before each week
- We want to be sure to cover weekly "wins", but be careful of time for meeting efficiency
- Be careful of FYI type things that could be covered in an email
- We have to be careful about talking about open positions. We want all qualified candidates to apply, and give them an equitable shot based on their ability. We can't give special preference to anyone, just because we work with them.
- Please feel free to bring up ethical issues and have us talk about them as a group. If you don't feel comfortable bringing it up with the group, please bring it someone that will get it to Will.
- Focussed discussion will be on conda (see below), who wants to join? Adrianna and Jackie?
- We have 91 open bugs. Let's make a concerted effort to address this. Can we do this in the next few months?
- Erik will show what he's setup in py_env_create at this point
The last half-hour (10:30 - 11:00) can be used either for a pre-determined topic or set aside as a time that can be used if anything comes up that needs some longer discussion (especially if not everyone needs to be there).
Suggestion of a bug-squashing party: take 2 weeks for a few of us to try to get through as many bugs as possible.
Some thoughts on this:
- Consider having a daily stand-up during this period
- Try to drop other things from our schedule (including user support) for this period
- Good opportunity for pair programming, especially with newer SEs
- Probably start the two-week period with a group triage
Keith: it would also be good to have a similar User's Guide update party... maybe not as fun, but needed.
- Pair writing can also work well here: more fun, and gives essentially real-time review / editing.
Tentative scheduling: Do one of these in the first two weeks of April, and maybe another in July.
Erik wasn't able to identify a single commit that could be backed out to remove dask.
Adrianna suggested that you can probably just swap out use of dask arrays for numpy arrays in a couple of places: Dask arrays are just numpy arrays with overhead.
Plan is still to remove the Dask stuff as long as Erik can find an easy way to do that.
What Erik has done so far is to make Dask optional in the conda environment. But we could just remove Dask entirely... feeling is that we should.
It seemed to Erik like the conda issues were coming not just from Dask but also from some plotting stuff for plotting meshes (matplotlib and cartopy). That plotting of meshes is really nice, but it adds some complexity to the python environment, so Erik is making this plotting optional.
We could put the plotting functionality in tools/contrib, and then not worry about supporting it.
- We could have a plotting environment as a separate environment, or let users use their own python environment. Feeling is probably do the latter.
For some packages, we want / need to specify specific versions. e.g., for black, we want to be sure everyone is using the same version. For others, there can be some practical benefit to specifying a specific version.
Erik also proposes having a version of requirements that lists specific versions.
- Bill's concern is that, if we're not testing it regularly, it could be a pain to maintain.
- Erik wants to try adding it and see if it becomes painful or not.
See notes here: https://docs.google.com/document/d/1UkC26yS9Ij3iIRbem346wEvo9n14idnQ3XHauRZhEQI/edit#
- LMWG was a big success.
- Thanks for your presentations, Jackie, Erik, Ryan, & Sam.
- Congrats to Keith for work with CLM-U, really impressive group of students presenting work at the meeting.
- Erik made dev tag 118! (actually that was a small tag that Sam did)
- Sam L's making progress on merging LULCC datasets for CTSM5.2
- Sam R started in SE position
- ELM making progress on C-based harvest with FATES (CLM is still behind on this) E3SM #5429
- finished: return status checks in cohort deallocations #824
- fixed problem with IBM and Cray compilers not working with FATES
- Greg made a toy model to diagnose and solve this problem, also helped to read the compiler documentation
- Ryan and Adrianna working on C and N balance checking between FATES and CTSM - what is the best way to do this?
- Adrianna going to start implementing multi-column FATES
- Will: Next round of coupled model testing will start in March 2023. Besides using the main ctsm5.1_dev tag are there other features we want to test out (e.g. CTSM5.2 surface dataset)?
- Will: I saw that CAM is working on Cloud to ground lightning flash frequency passed to land model. I'm assuming we should be testing this out on the land side. As always... by whom and when? * Note, this may be more appropriate to discuss at a science meeting?
- Erik: Can I get a subgroup to discuss conda?
Sam L: To use the new surface datasets in CTSM 5.1, you need to make a couple of changes in the code:
- Fix for nlevurb changes
- A pft mask that is no longer present on the surface datasets
It sounds like Peter L is regenerating the PFT raw data again.
Sam R can post his changes that were needed.
Keith: for the CESM2 CMIP6 PI control run, we did a spinup with coupled model forcing.
Will: if the point of this is a smoke test to see if we can stop the Lab Sea from freezing, is it necessary to have this full coupled model-based land spinup?
Ndep status:
- Scientifically, we should have essentially same ndep whether we get it from CAM or internally, if CAM is running without chemistry
- Currently, for cplhist forcing, datm expects ndep, so CAM would need to provide that, unless we make some changes to turn that off.
- We think no changes are needed in CTSM to get the ndep in from atm.
See https://github.com/ESCOMP/CTSM/issues/1630 for some discussion on this.
Erik: Two competing requirements:
- It would be nice to only set the minimum versions needed (so can test with updated versions). But this doesn't guarantee that you're going to get a functional environment.
- So there's also a rationale for specifying a specific set of versions.
Erik thought that conda lock would help with this. But in practice, this hasn't been working for Erik on either cheyenne or izumi.
Part of the complexity is dask. Erik suggests having a separate environment with dask. We might have to back out the dask stuff from this.
- Jackie supports taking dask out of this regional subsetting.
- Dask may be helpful for high-resolution grids, but since we aren't doing that now, it's not clear that dask is needed... even if things take an hour or so, that's probably okay as a one-time thing... and for most cases, it would probably take much less than an hour.
- YAGNI
The problem with keeping this in is that it seems impractical to set up a working python environment right now.
Jackie has an environment that works currently. So for now we could use Jackie's environment if needed.
- We wouldn't change the main CTSM python environment; instead, we'd have a note associated with the regional subsetting script about the need to create a particular environment.
- Erik notes that this would make some testing not work. We could back out those tests temporarily.
Erik will look into how hard it would be to back out the dask stuff in order to determine the best path forward.
- Adrianna and I demonstrated the value of "Test Driven Development". You add a test, show that it fails, and then add the feature until the test passes. We did that and the test didn't fail -- which meant there was a problem with it.
- It's ground hog day
- Adrianna's NEON/FATES tag will be done before LMWG!
- LMWG next week already!
- E3SM#5106 (c-based harvest) (Testing. Ryan investigating p test failure for eca) 971 (C starvation) and 926 (mortality and productivity) (testing. Jessie. Implement soon) 910 (water balance grass FATES-Hyrdo): Greg to make changes and test. Long term
- 769 (leaf memory): Closed and split up into three separate tasks. 978 below (cohort trimming) Ryan plan to implement adaptive trimming so that for cohorts that are non trimmable.
- [ ]Testing Atkin respiration across tropics.
- LAI issue
-
Erik: Proposal for interface for subset_data to create mesh. Require --create-domain option (if you want a mask that isn't 1 everywhere). Domain files are no longer useful, but they are a way that we can get the mask. Will make it optional to do the plots. --create-mesh, --create-domain, will be required for --create-user-mods. So you either say "--create-domain" or something like "--land-everywhere", "--nomask", "--mask-active-everywhere". I'd like to tie it to the existing domain files for now as this will work, and eventually ESMF is going to have more tools for mesh files. Another way to do this is to use the mesh_modifier tool -- but still I need a valid mask for the region.
-
--create-domainonly needed for regional runs that include ocean to create masks, otherwise this can be a--land-everywhereflag - Otherwise, this PR will make MCT obsolete (with the exception of the domain file that we're still using for a land mask).
- This isn't ideas, but it seems like this is a short-term solution that ESMF will help support moving forward.
- Erik feels like this is a more expedient way to run regional cases, without fixing bugs in the mesh capabilities from regional script that Negin made.
- Erik will finish the regional PR and work on cleaning up the sparse grid capabilities is working too, see #1919 and #1731
- Next tag will remove MCT testing from CTSM
-
-
Erik: Subtle limitation with the mesh maker right now is that it will only work on regular lat/lon grids. This is a problem for CTSM/WRF grids.
- This will also be an ESMF task moving forward, but Sam Levis' seems to have a workaround using ncks, again see #1919
- Erik and Sam will meet to discuss this further.
- Erik: We've been talking about CESM grid aliases in CSEG because of mizuRoute. We plan to have mizuRoute grids spelled out for example: f09_f09_rHDMAlk_mg17. Is there still a need for tri-grids? And is there work where the atmosphere will run on a regular grid and CTSM on HRU's? We currently aren't testing tri-grids I think that's a problem.
- Likely a science discussion, when do we want to run land on different grid than the atmosphere?
- Erik will create an issue for this, but for now a simple fix would be to create a test where we run the atmosphere on SE and land on FV grid.
- Erik: Some things from the FATES meetings. FATES PR board. The pre-meeting to set the agenda looks useful.
- ctsm5.1.dev116! With PR's started back in July finally in!
- Sam and Erik resolved a bunch of SLIM test failures!
- Adrianna's pronunciation -- Ahh-drianna. Please correct us when we get it wrong. A workplace of respect and belonging means we get each other names right. This is especially important for people with unfamiliar names (people from foreign countries) or for example a transgender colleague who transitions at work.
- Erik: We have a mapping problem with the mizuRoute lake grid. We are also at a point where I need to work on the coupler to pass lake Precip and ET.
- Erik: mizuRoute fields to pass. When we pass lake Precip and ET, the QGWL field (which normally is: lake, wetland, and glacier) probably should be just glacier and wetland. Should I setup a new set of arrays for QGW or overuse QGWL for both purposes? It's likely clearer to have another set even though it will add additional logic.
- Will: DOC production and export, (issues #1458 & #1216) introduce a number of CTSM, river, and ocean science opportunities and SE challenges. I don't really know how we handle this without additional support? Marius will present and the BGCWG & we can take it from there.
- Will: Should we check in on the CTSM6 project board at this meeting periodically?
- Erik: We've been talking about CESM grid aliases in CSEG because of mizuRoute. We plan to have mizuRoute grids spelled out for example: f09_f09_rHDMAlk_mg17. Is there still a need for tri-grids? And is there work where the atmosphere will run on a regular grid and CTSM on HRU's? We currently aren't testing tri-grids I think that's a problem.
- Erik: Proposed a meeting with Bill/Jackie for next week to talk about meeting agendas
- Erik: Some things from the FATES meetings. FATES PR board. The pre-meeting to set the agenda looks useful.
Erik: When we pass lake Precip and ET, the QGWL field (which normally is: lake, wetland, and glacier) probably should be just glacier and wetland. Should I setup a new set of arrays for QGW or overuse QGWL for both purposes? It's likely clearer to have another set even though it will add additional logic.
Bill's gut feeling: Maybe easiest to use QGWL always, but sometimes the lake contribution is 0. This feels similar to what we do with glacier, where contributions to certain flux fields are 0 if we are / aren't running with dynamic glaciers.
Erik: one concern is that mizuRoute currently can't handle tracers.
Probably about monthly is the right frequency for this.
- LULCC Hackathon for LMWG on schedule and structure developing
- CTSM PR expectations
- How to reword 2a so as to not suggest precluding turn on FATERS by default? (Prescribing from data will always be "better")
- 2a. DON'T degrade or break other parts of the model (i.e., do no harm)
- Erik: What would we like to have done before the LMWG meeting?
- Erik: We've been talking about CESM grid aliases in CSEG because of mizuRoute. We plan to have mizuRoute grids spelled out for example: f09_f09_rHDMAlk_mg17. Is there still a need for tri-grids? And is there work where the atmosphere will run on a regular grid and CTSM on HRU's? We currently aren't testing tri-grids I think that's a problem.
Will clarifies that https://github.com/ESCOMP/CTSM/wiki/CTSM-PR-Expectations comes from requests from the community for a better sense of what the PR process / expectations are. This isn't meant to be a significant change to our existing process / expectations, but more writing down our existing process / expectations.
How to word 2a, currently worded as "DON'T degrade or break other parts of the model (i.e., do no harm)"
- We don't want things that substantially degrade the system, but in practice many changes will lead to degradations in some areas
- Decision: we'll take out "degrade"
Erik will try to wrap up some tags; suggests trying to get Adrianna's recent FATES branch in.
Sam R would like to have his crop calendar stuff done (though not critical that it be totally merged by the LMWG meeting).
Erik suggests having a more descriptive name for these surface datasets like "16PFTs_mixed"... probably not having "FATES" in the name because this isn't really FATES-specific.
Will asks if there will be changes to the wrapper script.
- Adrianna: yes: it will add flags to do this the new way; if not specified, it won't change how it was done before.
- Adrianna is back!
- Penelope is growing and on her way in the world. Future scientist or engineer?
- Teagan is officially part of ESCOMP and CTSM on github
- PR 971 needs testing in CTSM discussion for new dimension (LU-type and LU-type by PFT) and integration after E3SM5106
- In progress: 971 (C starvation) (Jessie); units descriptions: 969 (bsap description); 968 (bleaf units), E3SM#5106 (c-based harvest) (in testing ELM) up to date with 888 (c-based harvest)] deconflicted and up-to-date with API25; 769 (leaf memory): scientific testing ongoing; 964 (land-use-category history var) add new dimension for LU-type and LU-type by PFT (after E3SM5106)
- Calibration: no update
-
CTSM PR expectations
prioritization
- Add “refactor, reduce technical debt” to list for 1 CTSM PR expectations
- Discussion - bug fixes highlight biases, we should allow for this possibility
- Ryan and Greg provide significant time to testing CTSM-FATES for integration of PRs
- Erik: I removed Negin from all her assigned issues, and made a bunch of them a "next" to figure out how to handle. I want to suggest that we need a dedicated person for WRF-CTSM work and as such should note that WRF-CTSM is stopped until there is funding for that person.
- Erik: Mariana's PR breaks the perl build-namelist to the point I think we need to dig up the idea of moving to python sooner than later.
- Erik: Is run_neon.py --fates something hard to do?
- Erik: What would we like to have done before the LMWG meeting?
- Erik: In talking with Peter L. he thought we should have different columns for natural vegetation (primary and secondary forest, pasture, and grazeland). I think we should open up a discussion on this.
- Land use transitions with FATES, can we make some progress on theIs run technical side of passing information between the HLM and FATES?
- Erik: We've been talking about CESM grid aliases in CSEG because of mizuRoute. We plan to have mizuRoute grids spelled out for example: f09_f09_rHDMAlk_mg17. Is there still a need for tri-grids? And is there work where the atmosphere will run on a regular grid and CTSM on HRU's? We currently aren't testing tri-grids I think that's a problem.
- Erik: I'd like to establish some "norms" on how we plan on handling PR's (and issues as appropriate) so expectations for all are set. This would be things like when a new PR comes in, how soon we will assign someone to review it, how soon we will first schedule it, etc. There needs to be some flexibility, but also accountability and expectations on both our side and the author. One way to do this would be for me to come up with a straw man, and then bring in a few others for feedback. After we have a small group we bring it to this group and then to TSS and then CTSM science group.
- Erik: Things go from "that can be done anytime" to "needs to be done now" really quickly. One part of the problem is lack of transparency in priorities and sequencing of order (this thing needs to happen so X person can do Y which enables Z person to do ZZ). For our internal stuff maybe we can add a "Needed for:" attribute to issues and PR's, so we can start to track this. We have this for version milestones like CTSM5.2, maybe other things could use it.
- Erik: It's hard for us as individuals or groups to keep track of more than 3 things at once. I want to suggest a workflow where we each individually have three things we are working on. I'm not going to do anything with that other than for myself. But, as a group, for this meeting I'd like to have three goals we examine each work and work towards. The three goals I suggest right now are: CTSM3, CTSM5.2, and CTSM/FATES-v1. We should adjust to the three things that most of this group is working on. For each of these goals then we want to have a way to visualize how much progress we are making on that goal, with metrics to see progress and how far we have to go. Being able to see that we are making progress on a goal is motivating, adds interest, and feels good for all. I propose that we also highlight blockers and needs for the three goals.
- Erik: Suggestion for list for monthly meetings:
- Go through status on goals
- Go through project boards on each goal
- Go through high priority project board
- Go through high priority issues
- What do we need to accomplish next month?
- How did we do on this month?
- What are long term issues/priorities we need to bring up in TSS and/or CTSM-science?
- Erik: Suggestion for list of quarterly meetings:
- Go through all PR's and update status
- Go through issues that affect answers: label "bug impacts science"
- Long term priorities/issues that we need to bring up in TSS/CTSM-science?
Something the FATES group has been talking about is having a new dimension in the output for land use type.
- PR 971 needs testing in CTSM discussion for new dimension (LU-type and LU-type by PFT) and integration after E3SM5106
- In progress: 971 (C starvation) (Jessie); 964 (land-use-category history var) add new dimension for LU-type and LU-type by PFT (after E3SM5106)
- PR prioritization
- Add “refactor, reduce technical debt” to list for 1 CTSM PR expectations
- Discussion - bug fixes highlight biases, we should allow for this possibility
- Ryan and Greg provide significant time to testing CTSM-FATES for integration of PRs
Agreement that this will be back-burnered for now, since we don't have someone to work on these. Bill will look through these issues and decide which, if any, should be totally closed.
Erik has some experience putting together python build-namelists now for mizuRoute and SLIM.
It would be great to leverage Alper's ParamGen tool, which CAM is moving to.
Erik: would it be hard to add a --fates option to run_neon?
Adrianna: technically it's easy. The harder part is deciding what the defaults should be for FATES vs. non-FATES. For example, there are differences in whether you want a dominant PFT or all PFTs for a site: most FATES users would want a mix of PFTs.
Jackie: one problem with run_neon is that it's too easy to treat it as a black box; with FATES, we really want people to think about these things and not just go with the default option.
Will: the advantage of run_neon is that it simplifies things for new users. There's an option to set up the case and not run it; would it make sense to use run_neon to set things up initially and then let FATES users modify things from there?
Adrianna has tried to break this into pieces: first getting things functional at all; then we can come back and build on this later.
Erik presented a way that we could avoid duplication of the settings for each site. The downside would be that there are 3 instances of directories for each site.
Another option would be to have multiple user mods directories, either via multiple things in --user-mods-dirs or having something picked up via the compset – but that only works easily if we can key off of the compset or running via a script that sets the multiple --user-mods-dirs for you (because it's a pain to require the user to do that).
Another option is Adrianna's original suggestion of having a variable in the includes; this probably wouldn't be too hard if this variable can be limited to something that keys off of the compset.
Feeling is: let's move forward with something, even if it isn't perfect or where we want to end up long-term. This could either be what Adrianna has now or what Erik is suggesting. Then we can revisit this once we have a 3rd case and so have a better sense of what might be needed.
Erik: one issue is that the mesh class doesn't have tests. We ideally would have tests for this, as for other things.
Bill: agree that we want tests with things like this... but it's most efficient for the tests to be written at the same time as the code... it feels inefficient to go and try to add tests to this now, and this is likely to hold up this PR. So suggests deferring tests and just adding these if/when someone comes back to this code to change it. One other note is that we might be replacing this mesh stuff with a more general CESM-wide or ESMF-wide solution at some point.
Will suggests adding some documentation in this module mentioning that this is untested.
It may be that we should start with upcoming tags, since that's something important that all people want to be involved with.
We may want more deliberate thought about what makes it onto the agenda – including what piece of upcoming tags are vs. aren't discussed.
Bill: There is an outside SE (a Google SE) who wants to volunteer some time helping with CESM, particularly if there are some refactoring / code cleanup tasks that he can help with.
Some ideas:
- Probably start with simple bfb stuff
- Documentation: could find some area of the documentation that isn't well fleshed out; he could learn how to do the given thing then document it for others
- Bigger project: Convert Peter's tool from C to Python
- Bigger project: Refactoring photosynthesis
-
Hope everyone had a wonderful holiday break.
-
I thought the meeting on meetings with Beatrice was helpful
-
Erik: I think we should have some project meetings for longer term priorities on a monthly and quarterly basis. We might invite more people to these meetings.
Some big recommendations
- Try to keep this meeting to an hour
- Try to move prioritization to either the monthly TSS meetings or a separate monthly-ish meeting with more people present, so that more people have a voice in this.
-
General
-
Documents
-
Bugs/Issues
-
Tutorials
-
Development guides
CTSM Users:
CTSM Developer Team
-
Meetings
-
Notes
-
Editing documentation (tech note, user's guide)