Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pylint error fixes #766

Open
wants to merge 63 commits into
base: v1.0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
85a022e
pushing just to update
amandarichardsonn May 2, 2024
ffaa9b3
updates
amandarichardsonn May 3, 2024
8210991
pushing updates
amandarichardsonn May 7, 2024
6e595fb
pushing mypy errors
amandarichardsonn May 8, 2024
821ffe3
Merge branch 'develop' into smartsim-refactor
amandarichardsonn May 15, 2024
f2ba3a7
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
MattToast May 20, 2024
919e65a
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn May 21, 2024
0c0ab7b
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 21, 2024
f3a58d4
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast May 23, 2024
3ed3ef7
Merge branch 'develop' of https://github.com/CrayLabs/SmartSim into s…
amandarichardsonn May 23, 2024
e96f893
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 23, 2024
e31c837
Move SmartSim v0.X.Y test to `tests/_legacy` dir (#575)
MattToast May 24, 2024
2cefbcf
Merge branch 'develop' into smartsim-refactor
amandarichardsonn May 28, 2024
ce35746
Merge branch 'smartsim-refactor' of https://github.com/CrayLabs/Smart…
amandarichardsonn May 28, 2024
d105df6
Fix Inconsistency in run_arg Formatting (#588)
juliaputko May 28, 2024
1cf41b1
Ignore build docs on the smartsim-refactor branch (#604)
juliaputko May 29, 2024
89214d6
Renaming Model to Application (#579)
juliaputko May 30, 2024
f21b6f8
Removal of Factory Methods from Experiment API (#581)
juliaputko Jun 3, 2024
883824b
Renaming Orchestrator and Database to Feature Store (#597)
juliaputko Jun 4, 2024
b64af80
Job and MPMD Job Implementation (#603)
juliaputko Jun 7, 2024
d058213
BaseJobGroup, JobGroup, ColocatedJobGroup skeleton (#599)
amandarichardsonn Jun 11, 2024
edb8364
BatchSettings, LaunchSettings, Command, CommandList and LaunchCommand…
amandarichardsonn Jun 13, 2024
e9d8eca
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast Jun 26, 2024
7e3e92c
Add Type Check Back to Refactor CI (#630)
MattToast Jul 8, 2024
c2164ca
Compound Entity and Ensemble (#605)
MattToast Jul 10, 2024
9b511f1
Command Generation (#615)
juliaputko Jul 24, 2024
b965c3d
Link up SmartSim Refactor CI (#646)
MattToast Jul 24, 2024
df6ed98
Merge remote-tracking branch 'upstream/develop' into smartsim-refactor
MattToast Jul 25, 2024
bd21957
Dispatch Jobs to Launchers based on Settings (#620)
MattToast Jul 27, 2024
a2c1251
Fix test overwriting a test configuration file (#648)
MattToast Jul 31, 2024
77eaf4d
Simple Status Checking (#655)
MattToast Aug 9, 2024
e44e099
Rename Status Enums (#663)
MattToast Aug 9, 2024
52cd8ec
Refactor of Generation class and inject path into Launch process (#650)
amandarichardsonn Aug 13, 2024
ddde9c5
Move `smartsim.settings.dispatch` module under `smartsim._core` (#659)
juliaputko Aug 14, 2024
347458f
merge develop into smartsim-refactor
juliaputko Aug 14, 2024
bf348a0
remove where install, build_py and Distribution were used in setup.py
juliaputko Aug 15, 2024
7ddb12f
Merge develop into smartsim-refactor (#668)
juliaputko Aug 15, 2024
0569c05
Move `ShellLauncher` (#658)
juliaputko Aug 21, 2024
a0cc447
Removed Slurm specific methods from LaunchArguments (#662)
juliaputko Aug 22, 2024
f6928e5
Redis removal (#656)
juliaputko Aug 22, 2024
f9a86d9
Unit tests for ShellLauncher & additional shell launch context (#671)
amandarichardsonn Aug 28, 2024
ec7677a
Remove dead attributes in application (#673)
juliaputko Aug 29, 2024
5611a16
Poll Based Waiting for Job Completion (#670)
MattToast Aug 29, 2024
4a4f43a
Refine SmartSimEntity Interface (#688)
amandarichardsonn Aug 29, 2024
c1faadd
Experiment Can Stop Launched Jobs (#677)
MattToast Aug 30, 2024
0175b6b
Ensemble Refactor (#686)
juliaputko Sep 5, 2024
c169878
Remove Problematic Type Union (#694)
MattToast Sep 5, 2024
f49fde3
Camel Case to Snake Case (#711)
amandarichardsonn Sep 18, 2024
3bfdff9
Removals of EntityList, EntitySequence, JobManager, and Controller (#…
juliaputko Sep 19, 2024
f748789
Short task to update BatchSettings (#704)
amandarichardsonn Sep 20, 2024
4faf95c
Ensemble Move short-task (#714)
amandarichardsonn Sep 23, 2024
4d9ab27
Replace TaggedFilesHierarchy with os.walk and implement configure_dir…
amandarichardsonn Sep 27, 2024
dbf7b72
Allow for sequences into exp.start(), and unpack iterables (#712)
juliaputko Oct 4, 2024
2cbd3be
Runtime value checking of public API methods (#707)
juliaputko Oct 7, 2024
a39246e
Application.files refactor (#732)
amandarichardsonn Oct 16, 2024
9fb5e3c
pylint error fixes, and temporary pylint suppressions to be removed a…
juliaputko Oct 25, 2024
94f078d
resolve merge conflicts, fix pylint errors
juliaputko Oct 28, 2024
4701e8c
Fix typos in documentation (#758)
jsta Oct 29, 2024
7e68e63
Merge branch 'pylintssrf' of https://github.com/juliaputko/SmartSim i…
juliaputko Oct 29, 2024
8bb01e1
invalid name fix
juliaputko Oct 29, 2024
cd7e3c5
fixed invalid name
juliaputko Oct 29, 2024
78396cd
the last of the pylint fixes
juliaputko Oct 31, 2024
94f0608
Merge branch 'v1pylintfix' of github.com:juliaputko/SmartSim into pyl…
juliaputko Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/build_docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ on:
push:
branches:
- develop
branches-ignore:
- smartsim-refactor

jobs:
build_docs:
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/changelog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ name: enforce_changelog

on:
pull_request:
branches-ignore:
- smartsim-refactor
push:
branches:
- develop
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/run_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,8 @@ jobs:
make check-mypy

# TODO: Re-enable static analysis once API is firmed up
# - name: Run Pylint
# run: make check-lint
- name: Run Pylint
run: make check-lint

# Run isort/black style check
- name: Run isort
Expand Down Expand Up @@ -172,7 +172,7 @@ jobs:
run: |
echo "SMARTSIM_LOG_LEVEL=debug" >> $GITHUB_ENV
py.test -s --import-mode=importlib -o log_cli=true --cov=$(smart site) --cov-report=xml --cov-config=./tests/test_configs/cov/local_cov.cfg --ignore=tests/full_wlm/ -m ${{ matrix.subset }} ./tests

# Upload artifacts on failure, ignoring binary files
- name: Upload Artifact
if: failure()
Expand Down
2 changes: 2 additions & 0 deletions doc/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ Description

- Implement workaround for Tensorflow that allows RedisAI to build with GCC-14
- Add instructions for installing SmartSim on PML's Scylla
- Fix typos in documentation

Detailed Notes

Expand All @@ -64,6 +65,7 @@ Detailed Notes
have yet to be installed at a system-wide level. Scylla has
its own entry in the documentation.
([SmartSim-PR733](https://github.com/CrayLabs/SmartSim/pull/733))
- Fix typos in the `train_surrogate` tutorial documentation


### 0.8.0
Expand Down
4 changes: 2 additions & 2 deletions doc/tutorials/ml_training/surrogate/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@

In this example, a neural network is trained to act like a surrogate model and to solve a
well-known physical problem, i.e. computing the steady state of heat diffusion. The training
dataset is constructed by running simualations *while* the model is being trained.
dataset is constructed by running simulations *while* the model is being trained.

The notebook also displays how the surrogate model prediction improves during training.
The notebook also displays how the surrogate model prediction improves during training.
2 changes: 1 addition & 1 deletion doc/tutorials/ml_training/surrogate/train_surrogate.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"\n",
"In this notebook, a neural network is trained to act like a surrogate model and to solve a\n",
"well-known physical problem, i.e. computing the steady state of heat diffusion. The training\n",
"dataset is constructed by running simualations *while* the model is being trained.\n",
"dataset is constructed by running simulations *while* the model is being trained.\n",
"\n",
"## 2D Heat Diffusion and Steady State\n",
"Throughout this notebook, the heat equation will be solved on the two-dimensional domain $[0,1]\\times[0,1]$, setting the initial temperature to 0 K, except\n",
Expand Down
24 changes: 13 additions & 11 deletions smartsim/_core/_cli/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,10 @@
import importlib.metadata
import operator
import os
import platform
from pathlib import Path
import re
import shutil
import textwrap
import typing as t
from pathlib import Path

from tabulate import tabulate

Expand All @@ -44,10 +42,8 @@
display_post_install_logs,
install_dragon,
)
from smartsim._core._cli.utils import SMART_LOGGER_FORMAT, pip
from smartsim._core._install import builder
from smartsim._core._install.buildenv import BuildEnv, SetupError, Version_, Versioner
from smartsim._core._install.builder import BuildError
from smartsim._core._cli.utils import SMART_LOGGER_FORMAT
from smartsim._core._install.buildenv import BuildEnv, Version_, Versioner
from smartsim._core._install.mlpackages import (
DEFAULT_MLPACKAGE_PATH,
DEFAULT_MLPACKAGES,
Expand All @@ -60,13 +56,17 @@
OperatingSystem,
Platform,
)
from smartsim._core._install.redisaiBuilder import RedisAIBuilder

from smartsim._core.config import CONFIG
from smartsim.error import SSConfigError
from smartsim.log import get_logger

logger = get_logger("Smart", fmt=SMART_LOGGER_FORMAT)

# ***************************************
# TODO: Remove pylint disable after merge
# ***************************************
# pylint: disable=too-many-statements,unused-variable,no-value-for-parameter,no-member,invalid-name,condition-evals-to-constant,unused-variable

# NOTE: all smartsim modules need full paths as the smart cli
# may be installed into a different directory.

Expand Down Expand Up @@ -143,7 +143,8 @@ def _format_incompatible_python_env_message(
conflict_str = fmt_list("Conflicting", conflicting)
sep = "\n" if missing_str and conflict_str else ""

return textwrap.dedent(f"""\
return textwrap.dedent(
f"""\
Python Package Warning:

Requested packages are missing or have a version mismatch with
Expand All @@ -153,7 +154,8 @@ def _format_incompatible_python_env_message(

Consider uninstalling any conflicting packages and rerunning
`smart build` if you encounter issues.
""")
"""
)


# pylint: disable-next=too-many-statements
Expand Down
6 changes: 0 additions & 6 deletions smartsim/_core/_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@
from smartsim._core._cli.clean import configure_parser as clean_parser
from smartsim._core._cli.clean import execute as clean_execute
from smartsim._core._cli.clean import execute_all as clobber_execute
from smartsim._core._cli.dbcli import execute as dbcli_execute
from smartsim._core._cli.info import execute as info_execute
from smartsim._core._cli.plugin import plugins
from smartsim._core._cli.site import execute as site_execute
Expand Down Expand Up @@ -118,11 +117,6 @@ def default_cli() -> SmartCli:
clean_execute,
clean_parser,
),
MenuItemConfig(
"dbcli",
"Print the path to the redis-cli binary",
dbcli_execute,
),
MenuItemConfig(
"site",
"Print the installation site of SmartSim",
Expand Down
1 change: 0 additions & 1 deletion smartsim/_core/_cli/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@

import argparse
import contextlib
import io
import os
import os.path
import tempfile
Expand Down
4 changes: 2 additions & 2 deletions smartsim/_core/_install/buildenv.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# pylint: disable=invalid-name

import importlib.metadata
import os
Expand Down Expand Up @@ -153,7 +152,8 @@ class Versioner:
``smart build`` command to determine which dependency versions
to look for and download.

Default versions for SmartSim, Redis, and RedisAI are specified here.
Default versions for SmartSim and its machine learning library dependencies
all defined here.
"""

# compatible Python version
Expand Down
6 changes: 0 additions & 6 deletions smartsim/_core/_install/builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,6 @@
from pathlib import Path
from subprocess import SubprocessError

from smartsim._core._install.platform import Architecture, OperatingSystem, Platform
from smartsim._core._install.utils import retrieve
from smartsim._core.utils import expand_exe_path

if t.TYPE_CHECKING:
from typing_extensions import Never

# TODO: check cmake version and use system if possible to avoid conflicts

Expand Down
2 changes: 2 additions & 0 deletions smartsim/_core/_install/platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,8 @@ class Device(enum.Enum):
CUDA12 = "cuda-12"
ROCM5 = "rocm-5"
ROCM6 = "rocm-6"
# TODO: check to see if this got overwritten/ gpu was removed
GPU = "gpu"

@classmethod
def from_str(cls, str_: str) -> "Device":
Expand Down
4 changes: 2 additions & 2 deletions smartsim/_core/commands/command.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,9 @@ def __len__(self) -> int:
"""Get the length of the command list."""
return len(self._command)

def insert(self, idx: int, value: str) -> None:
def insert(self, index: int, value: str) -> None:
"""Insert a command at the specified index."""
self._command.insert(idx, value)
self._command.insert(index, value)

def __str__(self) -> str: # pragma: no cover
string = f"\nCommand: {' '.join(str(cmd) for cmd in self.command)}"
Expand Down
7 changes: 4 additions & 3 deletions smartsim/_core/commands/command_list.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@ def __setitem__(
isinstance(item, str) for item in sublist.command
):
raise TypeError(
"Value sublists must be a list of Commands when assigning to a slice"
"Value sublists must be a list of Commands when assigning \
to a slice"
)
self._commands[idx] = (deepcopy(val) for val in value)

Expand All @@ -95,9 +96,9 @@ def __len__(self) -> int:
"""Get the length of the Command list."""
return len(self._commands)

def insert(self, idx: int, value: Command) -> None:
def insert(self, index: int, value: Command) -> None:
"""Insert a Command at the specified index."""
self._commands.insert(idx, value)
self._commands.insert(index, value)

def __str__(self) -> str: # pragma: no cover
string = "\n\nCommand List:\n\n"
Expand Down
6 changes: 2 additions & 4 deletions smartsim/_core/commands/launch_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,9 @@ def postlaunch_command(self) -> CommandList:

def __str__(self) -> str: # pragma: no cover
string = "\n\nPrelaunch Command List:\n"
for pre_cmd in self.prelaunch_command:
string += f"{pre_cmd}\n"
string += f"\n{' '.join(str(pre_cmd) for pre_cmd in self.prelaunch_command)}"
string += "\n\nLaunch Command List:\n"
for launch_cmd in self.launch_command:
string += f"{launch_cmd}\n"
string += f"\n{' '.join(str(launch_cmd) for launch_cmd in self.launch_command)}"
string += "\n\nPostlaunch Command List:\n"
for post_cmd in self.postlaunch_command:
string += f"{post_cmd}\n"
Expand Down
2 changes: 0 additions & 2 deletions smartsim/_core/config/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,6 @@

import psutil

from ...error import SSConfigError
from ..utils import expand_exe_path

# Configuration Values
#
Expand Down
3 changes: 2 additions & 1 deletion smartsim/_core/control/job.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ def __init__(self) -> None:

@property
def is_fs(self) -> bool:
"""Returns `True` if the entity represents a feature store or feature store shard"""
"""Returns `True` if the entity represents a feature store or feature
store shard"""
return self.type in ["featurestore", "fsnode"]

@property
Expand Down
1 change: 0 additions & 1 deletion smartsim/_core/dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@
from __future__ import annotations

import dataclasses
import os
import pathlib
import typing as t

Expand Down
3 changes: 2 additions & 1 deletion smartsim/_core/entrypoints/dragon_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,8 @@ def execute_entrypoint(args: DragonClientEntrypointArgs) -> int:

requests.append(DragonShutdownRequest(immediate=False, frontend_shutdown=True))

connector = DragonConnector()
# TODO: needs path
connector = DragonConnector(".")

for request in requests:
response = connector.send_request(request)
Expand Down
Loading