defer resolving helper models in self_critique() and `model_graded_…

…qa()`

CHANGELOG.md

@@ -14,6 +14,7 @@
 - Improve mime type detection for image content encoding (fixes issues w/ webp images). 
 - Fix memory leak in Inspect View worker-based JSON parsing.
 - Add `fail_on_error` option for `eval_retry()` and `inspect eval-retry`.
+- Defer resolving helper models in `self_critique()` and `model_graded_qa()`.
 
 ## v0.3.40 (6 October 2024)
 

examples/theory_of_mind.py

@@ -1,4 +1,4 @@
-from inspect_ai import Task, task
+from inspect_ai import Task, eval, task
 from inspect_ai.dataset import example_dataset
 from inspect_ai.scorer import model_graded_fact
 from inspect_ai.solver import chain_of_thought, generate, self_critique
@@ -16,3 +16,7 @@ def theory_of_mind(critique=False):
         solver=solver,
         scorer=model_graded_fact(),
     )
+
+
+if __name__ == "__main__":
+    eval(theory_of_mind(critique=True), model="openai/gpt-4o")