Pull jcsda-emc libs and repos out of spack fork

[AlexanderRichert-NOAA]

Summary

This PR takes lib/jcsda-emc/, var/spack/repos/jcsda-emc/, and var/spack/repos/jcsda-emc-bundles/ out of our spack fork and places them under spack-stack in subdirectory spack-ext. setup.sh has been modified so that when run for the first time in a new spack-stack instance, the extensions and repos will be added to the spack default configuration (etc/spack/defaults/{config.yaml,repos.yaml}). This will make it easier to swap in copies of main spack for testing purposes, as well as make it easier to compare and minimize differences with main spack.

Testing

Tested on personal machine. setup.sh should definitely be tested on other platforms, especially macOS.

Applications affected

n/a

Systems affected

all

Dependencies

JCSDA/spack#394

Issue(s) addressed

Addresses #436

Checklist

• This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
• These changes have been tested on the affected systems and applications.
• All dependency PRs/issues have been resolved and this PR can be merged.

[climbfuji]

This is neat. Question: If you run setup.sh as is, will it update your ~/.spack config or where does spack "store" this configuration?

[AlexanderRichert-NOAA]

For both the extensions and the repos, it updates the config.yaml and repos.yaml, respectively, in etc/spack/defaults/. This is set by the "--scope" option.

[climbfuji]

For both the extensions and the repos, it updates the config.yaml and repos.yaml, respectively, in etc/spack/defaults/. This is set by the "--scope" option.

I should have noticed that ... thanks! Let me give this a try on macOS

[climbfuji]

This works great! A couple of comments:
1. There are some left over print statements that can be removed:

...
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
Configuring basic directory information ...
  ... script directory: /Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/lib/jcsda-emc/spack-stack/stack
  ... base directory: /Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/lib/jcsda-emc/spack-stack
  ... spack directory: /Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
Creating environment from command-line args
/Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/spack
Successfully wrote environment at /Users/heinzell/work/spack-stack/spack-stack-alex-separate-spack/envs/unified-env/spack.yaml

2. Can we (should we?) move lib and repos under a subdirectory spack-ext ?

[AlexanderRichert-NOAA]

1. Done.
2. If we anticipate having more directories as we pull out our Spack modifications, then I'd say yes, but if not then I have no preference. For the moment I can't think of any other directories we would need to create down the road.

[climbfuji]

1. Done.
2. If we anticipate having more directories as we pull out our Spack modifications, then I'd say yes, but if not then I have no preference. For the moment I can't think of any other directories we would need to create down the road.

Ok, thanks. I was thinking that users who deploy spack-stack on their systems may find the current directory layout confusing; it's easier to understand if the stuff that is part of the spack extension in spack-stack lives in a directory that's labeled as such?

[climbfuji]

1. Done.
2. If we anticipate having more directories as we pull out our Spack modifications, then I'd say yes, but if not then I have no preference. For the moment I can't think of any other directories we would need to create down the road.

Ok, thanks. I was thinking that users who deploy spack-stack on their systems may find the current directory layout confusing; it's easier to understand if the stuff that is part of the spack extension in spack-stack lives in a directory that's labeled as such?

I can suggest changes for your PR if you like so that you don't have to code it up (should be trivial I suppose).

[AlexanderRichert-NOAA]

Just made the change. I agree this directory structure makes sense.

[climbfuji]

Thanks very much! And it works just fine on macOS.

One tiny little caveat that we developers need to remember with this change is that git diff will show that spack has been modified even though we didn't make any actual code changes - just need to be careful to not commit those changes back accidentally.

Thanks for doing this!

[climbfuji]

I added a bunch of reviewers here, more for their awareness than for requiring their reviews (but please review if you feel the desire to do so!).

[climbfuji]

I merged the spack PR, new hash is JCSDA/spack@e7ef791