Skip to content

Fixed invalid JSON format from the logs #588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fixed invalid JSON format from the logs #588

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c873f96
Changed saving and loading function so the logs can be saved as valid…
Fdms-3741 Mar 26, 2025
1ed36e9
Updated
Fdms-3741 Aug 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion bayes_opt/logger.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,18 @@ def log_optimization_step(
result : dict[str, Any]
The result dictionary for the most recent step.

params_config : Mapping[str, ParamsType]
# Read current data
with self._path.open("r") as f:
fileData = json.load(f)

# Append next data point
fileData.append(data)

# Writes content back to a file
with self._path.open("w") as f:
json.dumps(fileData)

params_config: Mapping[str, ParamsType]
Comment on lines +274 to +285
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove stray JSON I/O pseudo-code from the docstring; it’s misleading and incorrect

This block references undefined identifiers (self._path, data), and json.dumps() is called without writing to the file. It also interrupts the Parameters section. If you want to document JSON logging, put it under the appropriate file-logger class, not ScreenLogger.

Apply this diff to drop the block:

-            # Read current data
-            with self._path.open("r") as f:
-                fileData = json.load(f)
-
-            # Append next data point
-            fileData.append(data)
-
-            # Writes content back to a file
-            with self._path.open("w") as f:
-                json.dumps(fileData)
-
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Read current data
with self._path.open("r") as f:
fileData = json.load(f)
# Append next data point
fileData.append(data)
# Writes content back to a file
with self._path.open("w") as f:
json.dumps(fileData)
params_config: Mapping[str, ParamsType]
params_config: Mapping[str, ParamsType]
🤖 Prompt for AI Agents 
In bayes_opt/logger.py around lines 274 to 285, remove the stray JSON I/O
pseudo-code block (references to self._path, data, and the incorrect
json.dumps() call) from the ScreenLogger docstring so it no longer interrupts
the Parameters section; if JSON logging needs documenting, add a concise,
correct example under the dedicated file-logger class or module docs and, when
showing real code, use json.dump(fileData, f) to write to the file.

The configuration to map the key to the parameter for correct formatting.

current_max : dict[str, Any] | None
Expand Down
50 changes: 50 additions & 0 deletions bayes_opt/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,58 @@

from __future__ import annotations

import json
from collections.abc import Iterable
from os import PathLike
from pathlib import Path

import numpy as np

# from bayes_opt.bayesian_optimization import BayesianOptimization

Comment on lines +5 to +13
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix undefined type reference and imports; address Ruff F821 and future-proof typing

Ruff flags BayesianOptimization as undefined. Import it under TYPE_CHECKING to avoid runtime deps/cycles. Also import NotUniqueError for duplicate handling and warnings for diagnostics.

+import warnings
 import json
 from collections.abc import Iterable
 from os import PathLike
 from pathlib import Path
 
 import numpy as np
-# from bayes_opt.bayesian_optimization import BayesianOptimization
+from typing import TYPE_CHECKING
+from bayes_opt.target_space import NotUniqueError
+
+if TYPE_CHECKING:
+    from bayes_opt.bayesian_optimization import BayesianOptimization
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
import json
from collections.abc import Iterable
from os import PathLike
from pathlib import Path
import numpy as np
# from bayes_opt.bayesian_optimization import BayesianOptimization
import warnings
import json
from collections.abc import Iterable
from os import PathLike
from pathlib import Path
import numpy as np
from typing import TYPE_CHECKING
from bayes_opt.target_space import NotUniqueError
if TYPE_CHECKING:
from bayes_opt.bayesian_optimization import BayesianOptimization
🤖 Prompt for AI Agents 
In bayes_opt/util.py around lines 5 to 13, Ruff reports an undefined reference
to BayesianOptimization and you also need NotUniqueError and warnings for
duplicate handling/diagnostics; add "from typing import TYPE_CHECKING" and move
the BayesianOptimization import into an if TYPE_CHECKING: block so it’s only
used for type hints (avoids runtime deps/cycles), and add imports for
NotUniqueError (e.g., from sqlalchemy.exc import NotUniqueError) and warnings at
the top of the file so duplicate errors and warnings can be handled.


def load_logs(
optimizer: BayesianOptimization, logs: str | PathLike[str] | Iterable[str | PathLike[str]]
) -> BayesianOptimization:
"""Load previous ...

Parameters
----------
optimizer : BayesianOptimizer
Optimizer the register the previous observations with.

logs : str or os.PathLike
File to load the logs from.

Returns
-------
The optimizer with the state loaded.

"""
if isinstance(logs, (str, PathLike)):
logs = [logs]

for log in logs:
try:
with Path(log).open("r") as fil:
fileData = json.load(fil)
except json.JSONDecodeError:
print(f"ERROR: JSON decode error when decoding '{log}'")
continue

for iteration in fileData:
# Prevents duplicate points being registered when an exception can be raised
if not optimizer._allow_duplicate_points and iteration["params"] in optimizer:
continue

optimizer.register(
params=iteration["params"],
target=iteration["target"],
constraint_value=(iteration["constraint"] if optimizer.is_constrained else None),
)

return optimizer
Comment on lines +33 to +55
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Refactor duplicate detection and error handling; avoid invalid membership test and improve robustness

  • iteration["params"] in optimizer is unsafe unless BayesianOptimization implements contains. Rely on TargetSpace.register to detect duplicates and catch NotUniqueError.
  • Handle missing keys and constraint values gracefully.
  • Prefer warnings over prints; handle file I/O errors and non-list JSON roots.
  • Use snake_case variable names and explicit UTF-8.
     if isinstance(logs, (str, PathLike)):
         logs = [logs]
 
     for log in logs:
-        try:
-            with Path(log).open("r") as fil:
-                fileData = json.load(fil)
-        except json.JSONDecodeError:
-            print(f"ERROR: JSON decode error when decoding '{log}'")
-            continue
-
-        for iteration in fileData:
-            # Prevents duplicate points being registered when an exception can be raised
-            if not optimizer._allow_duplicate_points and iteration["params"] in optimizer:
-                continue
-
-            optimizer.register(
-                params=iteration["params"],
-                target=iteration["target"],
-                constraint_value=(iteration["constraint"] if optimizer.is_constrained else None),
-            )
+        path = Path(log)
+        try:
+            with path.open("r", encoding="utf-8") as fil:
+                file_data = json.load(fil)
+        except json.JSONDecodeError:
+            warnings.warn(f"Skipping '{path}': invalid JSON", RuntimeWarning)
+            continue
+        except OSError as e:
+            warnings.warn(f"Skipping '{path}': {e}", RuntimeWarning)
+            continue
+
+        if not isinstance(file_data, list):
+            warnings.warn(f"Skipping '{path}': expected a JSON array of iterations", RuntimeWarning)
+            continue
+
+        for iteration in file_data:
+            try:
+                params = iteration["params"]
+                target = iteration["target"]
+                constraint_value = iteration.get("constraint") if optimizer.is_constrained else None
+                optimizer.register(params=params, target=target, constraint_value=constraint_value)
+            except KeyError as e:
+                warnings.warn(f"Skipping malformed iteration in '{path}': missing key {e}", RuntimeWarning)
+                continue
+            except NotUniqueError:
+                # Duplicate point; skip when duplicates are not allowed
+                continue
+            except ValueError as e:
+                # e.g., missing constraint value when constrained
+                warnings.warn(f"Skipping iteration from '{path}': {e}", RuntimeWarning)
+                continue
 
     return optimizer
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if isinstance(logs, (str, PathLike)):
logs = [logs]
for log in logs:
try:
with Path(log).open("r") as fil:
fileData = json.load(fil)
except json.JSONDecodeError:
print(f"ERROR: JSON decode error when decoding '{log}'")
continue
for iteration in fileData:
# Prevents duplicate points being registered when an exception can be raised
if not optimizer._allow_duplicate_points and iteration["params"] in optimizer:
continue
optimizer.register(
params=iteration["params"],
target=iteration["target"],
constraint_value=(iteration["constraint"] if optimizer.is_constrained else None),
)
return optimizer
if isinstance(logs, (str, PathLike)):
logs = [logs]
for log in logs:
path = Path(log)
try:
with path.open("r", encoding="utf-8") as fil:
file_data = json.load(fil)
except json.JSONDecodeError:
warnings.warn(f"Skipping '{path}': invalid JSON", RuntimeWarning)
continue
except OSError as e:
warnings.warn(f"Skipping '{path}': {e}", RuntimeWarning)
continue
if not isinstance(file_data, list):
warnings.warn(f"Skipping '{path}': expected a JSON array of iterations", RuntimeWarning)
continue
for iteration in file_data:
try:
params = iteration["params"]
target = iteration["target"]
constraint_value = iteration.get("constraint") if optimizer.is_constrained else None
optimizer.register(
params=params,
target=target,
constraint_value=constraint_value,
)
except KeyError as e:
warnings.warn(f"Skipping malformed iteration in '{path}': missing key {e}", RuntimeWarning)
continue
except NotUniqueError:
# Duplicate point; skip when duplicates are not allowed
continue
except ValueError as e:
warnings.warn(f"Skipping iteration from '{path}': {e}", RuntimeWarning)
continue
return optimizer



def ensure_rng(random_state: int | np.random.RandomState | None = None) -> np.random.RandomState:
"""Create a random number generator based on an optional seed.
Expand Down
Loading