Automatic developer environment creation #470

teald · 2024-12-20T17:40:59Z

Summary

This PR creates a new, automated development environment for DRAGONS using nox. It works with venv and conda, following the current steps outlined in README.md + some quality of life checks.

It also introduces pre-commit hooks to the repository, which:

Format code (including MarkDown and Python files)
Check for large files being committed
Uniform file endings
Trim trailing whitespace
Other tidying things

Other changes of note

This PR serves as the initial formatting/lint of the entire codebase
- These commits are labeled with chore([lint|format]): and are added to .git-blame-ignore-revs to preserve our commit history for, e.g., git blame
Adds new developer documentation outlining our development toolkit, requirements, and other important notes for those developing with DRAGONS
This is the start of the process for migrating testing automations from tox to nox
- This adds native automation conda environment support, pure-python automation files, and better environment/build control and resolution without all the tox baggage
Some chores are folded in, specifically:
- Moving archived files from the main repository into archived/ directory

TODO

Add developer shell instructions
- README.md
- Developer Documentation
Developer documentation
- Add guide for making a branch conformant that is not yet using the formatting/linter

codecov · 2024-12-20T23:12:28Z

Codecov Report

Attention: Patch coverage is 87.17949% with 5 lines in your changes missing coverage. Please review.

Project coverage is 63.83%. Comparing base (a9c483d) to head (b027496).

Files with missing lines	Patch %	Lines
gemini_instruments/flamingos/adclass.py	0.00%	2 Missing ⚠️
gemini_instruments/igrins/adclass.py	0.00%	1 Missing ⚠️
geminidr/ghost/polyfit/polyspect.py	0.00%	1 Missing ⚠️
geminidr/gmos/lookups/bpmtab.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #470      +/-   ##
==========================================
- Coverage   63.88%   63.83%   -0.05%     
==========================================
  Files         314      314              
  Lines       38964    38985      +21     
==========================================
- Hits        24892    24887       -5     
- Misses      14072    14098      +26

Flag	Coverage Δ
f2	`25.84% <41.02%> (-0.03%)`	⬇️
ghost	`30.55% <41.02%> (-0.03%)`	⬇️
gmos	`29.32% <38.46%> (-0.03%)`	⬇️
gmosls	`35.47% <38.46%> (-0.03%)`	⬇️
gnirs	`30.61% <41.02%> (-0.03%)`	⬇️
gsaoi	`17.32% <38.46%> (-0.02%)`	⬇️
niri	`26.03% <38.46%> (-0.03%)`	⬇️
regression	`32.09% <84.61%> (-0.03%)`	⬇️
slow	`50.23% <41.02%> (-0.04%)`	⬇️
unit	`46.80% <41.02%> (-0.04%)`	⬇️
wavecal	`24.63% <41.02%> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Taking this out, since it seems more of a liability than anything that needs to be supported right now. Plus, it's probably better to have `nox -s devenv` the same for everyone who uses it.

This adds a placeholder entry in the pyproject.toml, which will be replaced with the dependency manager before merging.

These are default hooks that come with pre-commit.

Most changes are trailing lines. There are some changes to docstrings to either add them (in cases where header comments were used) or rectify non-conformant python code (e.g., tabbing issues). REBASE: Deferred to master branch.

This session just run pre-commit, which should be handling all linting/formatting/etc.

REBASE: Deferred to master branch

It's about to be deprecated, and it's useful to have tomllib around.

This is to provide all pythons to `nox`; if a user doesn't have a python version installed, they cannot make an environment with the correct one.

Ensure environment exists before attempting removal, skip if it does exist.

teald self-assigned this Dec 20, 2024

teald added 23 commits December 23, 2024 09:33

ci(nox): Initial noxfile.

6980c94

ci(nox): Add venv command to session

c14bc33

ci(nox): Pass positional arguments to venv if given.

83d0f78

ci(nox): Add catch for venv prompt w/o venv path

c3b6e1e

ci(nox): Extract venv's path into variable.

5f73521

ci(nox): Extract venv creation to helper function.

a715a6f

ci(nox): Remove fluff for custom dirs/prompts

8a7b92c

Taking this out, since it seems more of a liability than anything that needs to be supported right now. Plus, it's probably better to have `nox -s devenv` the same for everyone who uses it.

ci(nox): Add arguments to venv, install package

b6c4a68

ci(nox): Add dep to pyproject.toml, read in nox

fb5607e

This adds a placeholder entry in the pyproject.toml, which will be replaced with the dependency manager before merging.

doc(nox): Initial developer documentation

deef281

fix(nox): Open pyproject.toml in binary read mode

316e0ef

fix(nox): Use correct install method for venvs

09b7ff4

test(nox): Add tests for dev envs

93a1d51

ci(nox): Add devconda session

04c6d0a

ci(nox): Add initial devconda session impl

df62a11

test(nox): Testing for devconda environment

db9804c

ci(nox): Add devconda removal session.

396ced9

fix(nox): Add python specification to command if not an arg

efb416c

ci(nox): Add dependency installation for devconda.

cb8c618

test(nox): Test devconda dependency install

9e51c3b

ci(nox): Add pre-commit installation.

29e7ca4

docs(dev): Add to dev docs.

b9aa1c6

ci(nox): Add python version validation.

482223b

teald added 29 commits December 23, 2024 09:33

ci(nox): Add external flag to python version call.

47ff7cc

ci(pre-commit): Add initial pre-commit hooks

0326049

These are default hooks that come with pre-commit.

ci(nox): Add development tests to nox.

25781df

docs(ci): Add initial CI docs page.

0162998

ci(actions): Add dev tests workflow.

ab1d967

chore: Archive old_ directories.

c8d84b5

docs(ci): Rewording

88e6730

chore(lint): Initial link with pre-commit hooks

6b49b3d

Most changes are trailing lines. There are some changes to docstrings to either add them (in cases where header comments were used) or rectify non-conformant python code (e.g., tabbing issues). REBASE: Deferred to master branch.

ci(nox): Add noc lint session

736ecf1

This session just run pre-commit, which should be handling all linting/formatting/etc.

chore(lint): Apply mdformatter to *.md files.

ea5ce28

REBASE: Deferred to master branch

fix(docs): Links with trailing underscore

18b3b0a

ci(blame): Update git blame ignored commits.

8ac4ea4

ci(blame): Update .git-blame-ignore-revs after rebase.

075df89

test(dev): Update github workflow nox session

557b683

ci: Remove defaults channel

61ef758

chore(nox): Update quotes.

6571a39

ci(dev): Remove python 3.10

152ba41

It's about to be deprecated, and it's useful to have tomllib around.

ci(lint): Add initial linting workflow

e256e7b

ci(lint): Add python version specification to action

27f1940

ci(dev): Update test to use nox actions

18ba4be

This is to provide all pythons to `nox`; if a user doesn't have a python version installed, they cannot make an environment with the correct one.

ci(pre-commit): Update pre-commit repo

8ab9a5a

ci(lint): Rename workflow

7755535

ci(pre-commit): Update pre-commit config, add lint/format (off)

12d7a87

chore(README): Format README.

b9fd06d

ci(dev): Update python, conda installs.

c2efc08

ci(unit): Update unit test name

3106741

fix(ci): Add nox install to dev tests

ffebc44

test(dev): Update conda environment removal

2867381

Ensure environment exists before attempting removal, skip if it does exist.

test(devenv): Add tests for obs_db/calmgr deps

b027496

teald force-pushed the feature/developer-environment branch from b32d74c to b027496 Compare December 23, 2024 17:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automatic developer environment creation #470

Automatic developer environment creation #470

teald commented Dec 20, 2024 •

edited

Loading

codecov bot commented Dec 20, 2024 •

edited

Loading

Automatic developer environment creation #470

Are you sure you want to change the base?

Automatic developer environment creation #470

Conversation

teald commented Dec 20, 2024 • edited Loading

Summary

Other changes of note

TODO

codecov bot commented Dec 20, 2024 • edited Loading

Codecov Report

teald commented Dec 20, 2024 •

edited

Loading

codecov bot commented Dec 20, 2024 •

edited

Loading