From c70f53945bf8aec0a028a977ee6f2a53e7c148e0 Mon Sep 17 00:00:00 2001 From: aisi-inspect <166920645+aisi-inspect@users.noreply.github.com> Date: Tue, 12 Nov 2024 19:49:12 +0000 Subject: [PATCH] docs and changelog for release --- CHANGELOG.md | 2 +- docs/eval-logs.qmd | 6 ++---- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 76589a6b4..484e219fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -## Unreleased +## v0.3.46 (12 November 2024) - [eval](https://inspect.ai-safety-institute.org.uk/eval-logs.html#sec-log-format) is now the default log format (use `--log-format=json` to use old format). - Base 64 images are now logged by default for all log formats (disable with `--no-log-images`). diff --git a/docs/eval-logs.qmd b/docs/eval-logs.qmd index 3ef19e15d..79484730c 100644 --- a/docs/eval-logs.qmd +++ b/docs/eval-logs.qmd @@ -68,7 +68,7 @@ Both formats are fully supported by the [Log File API](#sec-log-file-api) and [L ### Format Option -The PyPI version of Inspect currently defaults to using the `.json` format, whereas the development version on GitHub currently defaults to `.eval` because of its size and speed advantages. You can explicitly control the log format globally in your `.env` file: +Beginning with Inspect v0.3.46, `.eval` is the default log file format. You can explicitly control the global log format default in your `.env` file: ``` {.bash filename=".env"} INSPECT_LOG_FORMAT=eval @@ -90,9 +90,7 @@ If you do need to interact with the underlying JSON (e.g., when reading logs fro ## Image Logging -When running the development version of Insepct from GitHub, full base64 encoded copies of images are included in the log file by default (the PyPI version currently still defaults to not logging images for the `.json` format). - -Image logging will not create performance problems when using `.eval` logs, however if you are using `.json` logs then large numbers of images could become unweildy (i.e. if your `.json` log file grows to 100mb or larger as a result). +By default, full base64 encoded copies of images are included in the log file. Image logging will not create performance problems when using `.eval` logs, however if you are using `.json` logs then large numbers of images could become unweildy (i.e. if your `.json` log file grows to 100mb or larger as a result). You can disable this using the `--no-log-images` flag. For example, here we enable the `.json` log format and disable image logging: