nidap_dashboards_repo_organization

NIDAP Code Dashboards Repository Organization

I will ask Palantir the best way to utilize two repositories simultaneously in Code Workspaces (one for Palantir-specific code and one for the repository of interest such as for analysis). Constraints include that Palantir typically wants conda-based packages as opposed to raw source code; that there is no internet access; and that utilizing two repositories simultaneously (even through git submodules) may be nuanced. In the meantime, we should keep both Palantir code and analysis code in the same repository. This is not best practices but provides the easiest solution (below) and good practices can still be followed such as separating out the Palantir code from the analysis code as much as possible. We can always (and will in the future) separate the Palantir and analysis repositories from each other but for the time being it is best to get something effective working quickly.

The following utilizes git commands on the command line. GitHub Desktop can likely be used to perform commands analogous to those on the command line, if desired. However, it might be best to stick to using the command line.

Assumptions

The repository name is saved to a command line environment variable $REPO_NAME, e.g., REPO_NAME=nidap-spatial-interaction-tool.
The git remote URL is saved (instructions below) to a command line environment variable $GIT_REMOTE_URL_DMAP, e.g., GIT_REMOTE_URL_DMAP=https://andrew.weisman%40nih.gov:[email protected]/stemma/git/ri.stemma.main.repository.RANDOMSTRING2/$REPO_NAME.
Locally, there exists a potentially unversioned set of analysis code in the directory $ORIG_ANALYSIS_CODE_DIR, e.g., ORIG_ANALYSIS_CODE_DIR=/home/weismanal/original_analysis_codebase
A git remote URL for a project-specific repository is stored in $GIT_REMOTE_URL_PROJECT1, e.g., GIT_REMOTE_URL_PROJECT1=https://andrew.weisman%40nih.gov:[email protected]/stemma/git/ri.stemma.main.repository.RANDOMSTRING2/nidap-spatial-interaction-tool__tais

Create the repository on NIDAP

Do this by creating a new Code Workspace in a project directory within the nih-dashboards directory in the DMAP project folder; you will be automatically prompted to create an associated repository as well, which you should assign the name $REPO_NAME.

Once the repository is created, open it and copy its git remote URL, saving it to the environment variable $GIT_REMOTE_URL_DMAP as in the Assumptions above.

Populate the repository with existing analysis code

Clone the new repository locally using:

git clone $GIT_REMOTE_URL_DMAP

You will now have a directory called $REPO_NAME in your working directory.

Enter the repository directory and change the remote name to dmap (so-named for the repository being located in the DMAP project on NIDAP).

cd $REPO_NAME/
git remote rename origin dmap

Copy the analysis code to the repository and commit and push the changes to the repository in the DMAP project on NIDAP:

cp -rp $ORIG_ANALYSIS_CODE_DIR/* .
git add .
git commit -m "Initial commit of analysis code"
git push dmap master

"Copy" the repository to a specific project

Create an EMPTY repository on NIDAP in the specific project folder, such as TaIS. Call it the same as the main repository (such as nidap-spatial-interaction-tool) but with __PROJECTNAME appended to it, e.g., nidap-spatial-interaction-tool__tais.

Open the repository, delete its default file README.md (don't forget to "commit" that deletion!), and copy its git remote URL, saving it to the environment variable, e.g., $GIT_REMOTE_URL_PROJECT1 as in the Assumptions above.

Add the remote to the new, empty repository on NIDAP:

git remote add project1 $GIT_REMOTE_URL_PROJECT1

Force-push the codebase to the project-specific repository:

git push -f project1 master

Normal usage

You can now push/pull to/from the different remotes as usual and use branching as usual.

Obviously you can create multiple Workspaces and repositories for different projects and use them as additional remotes.

If you create a new repository on GitHub and add a remote to that repository (e.g., git remote add github GITHUBURL), you can use that as yet another remote and then utilize all the features GitHub offers.

Notes

You may have to update repositories that have been pushed to at the repositories' locations on NIDAP (by refereshing the repository when prompted in the blue banner).
Until the "bug" is nailed down further, we should make a habit of refreshing Workspaces (starting them and then restarting them) to ensure the latest code updates are pulled in to the Workspace.

TODO

Add workflow for preserving the history of an analysis repository. Not tested as the above is, but almost certainly we can do this by copying all template files in a new Workspaces repository (including hidden files/directories) to the analysis repository and then force-pushing the analysis repository to the Workspaces repository. (20230421_2100: Confirmed, this essentially works!)