Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core Refactor SmartSim API updates #768

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
85a022e
pushing just to update
amandarichardsonn May 2, 2024
ffaa9b3
updates
amandarichardsonn May 3, 2024
8210991
pushing updates
amandarichardsonn May 7, 2024
6e595fb
pushing mypy errors
amandarichardsonn May 8, 2024
821ffe3
Merge branch 'develop' into smartsim-refactor
amandarichardsonn May 15, 2024
f2ba3a7
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
MattToast May 20, 2024
919e65a
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn May 21, 2024
0c0ab7b
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 21, 2024
f3a58d4
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast May 23, 2024
3ed3ef7
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn May 23, 2024
e96f893
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 23, 2024
e31c837
Move SmartSim v0.X.Y test to `tests/_legacy` dir (#575)
MattToast May 24, 2024
2cefbcf
Merge branch 'develop' into smartsim-refactor
amandarichardsonn May 28, 2024
ce35746
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 28, 2024
d105df6
Fix Inconsistency in run_arg Formatting (#588)
juliaputko May 28, 2024
1cf41b1
Ignore build docs on the smartsim-refactor branch (#604)
juliaputko May 29, 2024
89214d6
Renaming Model to Application (#579)
juliaputko May 30, 2024
f21b6f8
Removal of Factory Methods from Experiment API (#581)
juliaputko Jun 3, 2024
883824b
Renaming Orchestrator and Database to Feature Store (#597)
juliaputko Jun 4, 2024
b64af80
Job and MPMD Job Implementation (#603)
juliaputko Jun 7, 2024
d058213
BaseJobGroup, JobGroup, ColocatedJobGroup skeleton (#599)
amandarichardsonn Jun 11, 2024
edb8364
BatchSettings, LaunchSettings, Command, CommandList and LaunchCommand…
amandarichardsonn Jun 13, 2024
e9d8eca
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast Jun 26, 2024
7e3e92c
Add Type Check Back to Refactor CI (#630)
MattToast Jul 8, 2024
c2164ca
Compound Entity and Ensemble (#605)
MattToast Jul 10, 2024
9b511f1
Command Generation (#615)
juliaputko Jul 24, 2024
b965c3d
Link up SmartSim Refactor CI (#646)
MattToast Jul 24, 2024
df6ed98
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast Jul 25, 2024
bd21957
Dispatch Jobs to Launchers based on Settings (#620)
MattToast Jul 27, 2024
a2c1251
Fix test overwriting a test configuration file (#648)
MattToast Jul 31, 2024
77eaf4d
Simple Status Checking (#655)
MattToast Aug 9, 2024
e44e099
Rename Status Enums (#663)
MattToast Aug 9, 2024
52cd8ec
Refactor of Generation class and inject path into Launch process (#650)
amandarichardsonn Aug 13, 2024
ddde9c5
Move `smartsim.settings.dispatch` module under `smartsim._core` (#659)
juliaputko Aug 14, 2024
347458f
merge develop into smartsim-refactor
juliaputko Aug 14, 2024
bf348a0
remove where install, build_py and Distribution were used in setup.py
juliaputko Aug 15, 2024
7ddb12f
Merge develop into smartsim-refactor (#668)
juliaputko Aug 15, 2024
0569c05
Move `ShellLauncher` (#658)
juliaputko Aug 21, 2024
a0cc447
Removed Slurm specific methods from LaunchArguments (#662)
juliaputko Aug 22, 2024
f6928e5
Redis removal (#656)
juliaputko Aug 22, 2024
f9a86d9
Unit tests for ShellLauncher & additional shell launch context (#671)
amandarichardsonn Aug 28, 2024
ec7677a
Remove dead attributes in application (#673)
juliaputko Aug 29, 2024
5611a16
Poll Based Waiting for Job Completion (#670)
MattToast Aug 29, 2024
4a4f43a
Refine SmartSimEntity Interface (#688)
amandarichardsonn Aug 29, 2024
c1faadd
Experiment Can Stop Launched Jobs (#677)
MattToast Aug 30, 2024
0175b6b
Ensemble Refactor (#686)
juliaputko Sep 5, 2024
c169878
Remove Problematic Type Union (#694)
MattToast Sep 5, 2024
f49fde3
Camel Case to Snake Case (#711)
amandarichardsonn Sep 18, 2024
3bfdff9
Removals of EntityList, EntitySequence, JobManager, and Controller (#…
juliaputko Sep 19, 2024
f748789
Short task to update BatchSettings (#704)
amandarichardsonn Sep 20, 2024
4faf95c
Ensemble Move short-task (#714)
amandarichardsonn Sep 23, 2024
a4cea3d
update smartsim API with core changes
juliaputko Sep 25, 2024
4d9ab27
Replace TaggedFilesHierarchy with os.walk and implement configure_dir…
amandarichardsonn Sep 27, 2024
0d7034e
Merge branch 'smartsim-refactor' of github.com:CrayLabs/SmartSim into…
juliaputko Oct 2, 2024
d72a97a
updated smartsimapi
juliaputko Oct 15, 2024
7ea1557
import settings, and title fixes in api docs
juliaputko Oct 24, 2024
d6902cb
path changes for docs
juliaputko Oct 25, 2024
392a95a
docs fix
juliaputko Oct 25, 2024
e618758
readthedocs error fixes
juliaputko Oct 25, 2024
2b30334
readthedocs error fixes
juliaputko Oct 25, 2024
0756a60
readthedocs error fixes
juliaputko Oct 26, 2024
8db1ff1
fix spacing issue
juliaputko Oct 28, 2024
abad5a8
resolve merge conflicts
juliaputko Oct 28, 2024
328ea86
unpin sphinx versions
juliaputko Oct 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/build_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ on:
push:
branches:
- develop
branches-ignore:
- smartsim-refactor

jobs:
build_docs:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/changelog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ name: enforce_changelog

on:
pull_request:
branches-ignore:
- smartsim-refactor
push:
branches:
- develop
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ jobs:
run: |
echo "SMARTSIM_LOG_LEVEL=debug" >> $GITHUB_ENV
py.test -s --import-mode=importlib -o log_cli=true --cov=$(smart site) --cov-report=xml --cov-config=./tests/test_configs/cov/local_cov.cfg --ignore=tests/full_wlm/ -m ${{ matrix.subset }} ./tests

# Upload artifacts on failure, ignoring binary files
- name: Upload Artifact
if: failure()
Expand Down
50 changes: 40 additions & 10 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,22 +55,51 @@
from smartsim._core.config.config import Config
from smartsim._core.launcher.dragon.dragon_connector import DragonConnector
from smartsim._core.launcher.dragon.dragon_launcher import DragonLauncher
from smartsim._core.generation.operations.operations import ConfigureOperation, CopyOperation, SymlinkOperation
from smartsim._core.generation.operations.operations import (
ConfigureOperation,
CopyOperation,
SymlinkOperation,
)
from smartsim._core.generation.generator import Generator
from smartsim._core.utils.telemetry.telemetry import JobEntity
from smartsim.database import FeatureStore
from smartsim.entity import Application
from smartsim.error import SSConfigError, SSInternalError
from smartsim.log import get_logger


# Mock imports
class AprunSettings: pass
class DragonRunSettings: pass
class JsrunSettings: pass
class MpiexecSettings: pass
class MpirunSettings: pass
class PalsMpiexecSettings: pass
class RunSettings: pass
class SrunSettings: pass
class AprunSettings:
pass


class DragonRunSettings:
pass


class JsrunSettings:
pass


class MpiexecSettings:
pass


class MpirunSettings:
pass


class PalsMpiexecSettings:
pass


class RunSettings:
pass


class SrunSettings:
pass


logger = get_logger(__name__)

Expand Down Expand Up @@ -473,6 +502,7 @@ def check_output_dir() -> None:
def fsutils() -> t.Type[FSUtils]:
return FSUtils


@pytest.fixture
def files(fileutils):
path_to_files = fileutils.get_test_conf_path(
Expand Down Expand Up @@ -526,6 +556,7 @@ def configure_operation(mock_src: pathlib.Path, mock_dest: pathlib.Path):
src=mock_src, dest=mock_dest, file_parameters={"FOO": "BAR"}
)


class FSUtils:
@staticmethod
def get_fs_configs() -> t.Dict[str, t.Any]:
Expand Down Expand Up @@ -1000,7 +1031,6 @@ def local_fs() -> t.Generator[DBConfiguration, None, None]:
_cleanup_fs(name)



@pytest.fixture(scope="session")
def single_fs(wlmutils: WLMUtils) -> t.Generator[DBConfiguration, None, None]:
hostlist = wlmutils.get_test_hostlist()
Expand Down
Loading
Loading