AzureAI: Use Model Inference API (preview) for implementation of mode…

…l client (#807)

* add azure-ai-inference to dev dependencies

* remove self-signed workaround

* initial work on azure inference api

* remove unused

* initial azureai tests

* use tool calling fallback for llama on azureai

* remove unused imports

* playback tool calls to model

* prevent azure logging

* update changelog

* increase context window; handle bad request errors

* tag as content_filter

---------

Co-authored-by: aisi-inspect <[email protected]>

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## Unreleased
 
+- AzureAI: Use Model Inference API (preview) for implementation of model client.
 - Fix issue with correctly logging task_args for eval-set tasks which are interrupted.
 - Move `INSPECT_DISABLE_MODEL_API` into `generate()` (as opposed to `get_model()`)
 - Always treat `.eval` files as logs (don't apply file name pattern restrictions as we do with `.json`).

pyproject.toml

@@ -113,6 +113,7 @@ inspect = "inspect_ai._cli.main:main"
 dev = [
     "anthropic", 
     "aioboto3", 
+    "azure-ai-inference",
     "google-cloud-aiplatform",
     "google-generativeai", 
     "groq", 

src/inspect_ai/_display/logger.py

@@ -55,7 +55,11 @@ def __init__(self, levelno: int, transcript_levelno: int) -> None:
     @override
     def emit(self, record: LogRecord) -> None:
         # demote httpx and return notifications to log_level http
-        if record.name == "httpx" or "Retrying request" in record.getMessage():
+        if (
+            record.name == "httpx"
+            or "http" in record.name
+            or "Retrying request" in record.getMessage()
+        ):
             record.levelno = HTTP
             record.levelname = HTTP_LOG_LEVEL
 

src/inspect_ai/_util/content.py

@@ -21,8 +21,7 @@ class ContentImage(BaseModel):
     detail: Literal["auto", "low", "high"] = Field(default="auto")
     """Specifies the detail level of the image.
 
-    Currently only supported for OpenAI. Learn more in the
-    [Vision guide](https://platform.openai.com/docs/guides/vision/low-or-high-fidelity-image-understanding).
+    Currently only supported for OpenAI. Learn more in the    [Vision guide](https://platform.openai.com/docs/guides/vision/low-or-high-fidelity-image-understanding).
     """
 
 

src/inspect_ai/model/_image.py

@@ -0,0 +1,15 @@
+from copy import copy
+
+from pydantic import JsonValue
+
+from inspect_ai._util.constants import BASE_64_DATA_REMOVED
+
+
+def image_url_filter(key: JsonValue | None, value: JsonValue) -> JsonValue:
+    # remove images from raw api call
+    if key == "image_url" and isinstance(value, dict) and "url" in value:
+        url = str(value.get("url"))
+        if url.startswith("data:"):
+            value = copy(value)
+            value.update(url=BASE_64_DATA_REMOVED)
+    return value