Make documentation consistent, and raise PermissionError for exec

[art-dsit]

This PR contains:

• New features
• Changes to dev-tools e.g. CI config / github tooling
• Docs
• Bug fixes
• Code refactor

What is the current behavior? (You can also link to an open issue here)

The sandbox interface docs are inconsistent.
Sandbox behaviour is inconsistent when you try to exec something that's not executable.

What is the new behavior?

Docs are consistent.
The docker sandbox checks for "permission denied", for consistency with local. (Note: you could argue that the current behaviour of the docker sandbox makes more sense, and instead we should change the local sandbox. Happy to change this PR to do that instead)

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

It's possible that someone is coupled to the current behaviour, but the code here is an edge case already, so seems unlikely.

Other information:

[jjallaire]

Thank you for cleaning up that messiness in the docs + other inconsistencies! Some other related changes we should make here:

1. Add the directory error to this type (maybe as "is_a_directory"):

   inspect_ai/src/inspect_ai/tool/_tool_call.py

   Lines 24 to 32 in 2ed9d67

   	class ToolCallError:
   	type: Literal[
   	"parsing",
   	"timeout",
   	"unicode_decode",
   	"permission",
   	"file_not_found",
   	"unknown",
   	]

2. Catch IsADirectoryError here and construct a ToolCallError("is_a_directory", ...):

   inspect_ai/src/inspect_ai/model/_call_tools.py

   Lines 65 to 90 in 2ed9d67

   	try:
   	with track_store_changes():
   	result = await call_tool(tdefs, call)
   	except TimeoutError:
   	tool_error = ToolCallError(
   	"timeout", "Command timed out before completing."
   	)
   	except UnicodeDecodeError as ex:
   	tool_error = ToolCallError(
   	"unicode_decode",
   	f"Error decoding bytes to {ex.encoding}: {ex.reason}",
   	)
   	except PermissionError as ex:
   	message = f"{ex.strerror}."
   	if isinstance(ex.filename, str):
   	message = f"{message} Filename '{ex.filename}'."
   	tool_error = ToolCallError("permission", message)
   	except FileNotFoundError as ex:
   	tool_error = ToolCallError(
   	"file_not_found",
   	f"File '{ex.filename}' was not found.",
   	)
   	except ToolParsingError as ex:
   	tool_error = ToolCallError("parsing", ex.message)
   	except ToolError as ex:
   	tool_error = ToolCallError("unknown", ex.message)

Note that we try to use known attributes of the exception to provide as detailed an error message as we can.

[art-dsit]

Thank you for cleaning up that messiness in the docs + other inconsistencies! Some other related changes we should make here:

1. Add the directory error to this type (maybe as "is_a_directory"):

   inspect_ai/src/inspect_ai/tool/_tool_call.py

   Lines 24 to 32 in 2ed9d67

   	class ToolCallError:
   	type: Literal[
   	"parsing",
   	"timeout",
   	"unicode_decode",
   	"permission",
   	"file_not_found",
   	"unknown",
   	]

2. Catch IsADirectoryError here and construct a ToolCallError("is_a_directory", ...):

   inspect_ai/src/inspect_ai/model/_call_tools.py

   Lines 65 to 90 in 2ed9d67

   	try:
   	with track_store_changes():
   	result = await call_tool(tdefs, call)
   	except TimeoutError:
   	tool_error = ToolCallError(
   	"timeout", "Command timed out before completing."
   	)
   	except UnicodeDecodeError as ex:
   	tool_error = ToolCallError(
   	"unicode_decode",
   	f"Error decoding bytes to {ex.encoding}: {ex.reason}",
   	)
   	except PermissionError as ex:
   	message = f"{ex.strerror}."
   	if isinstance(ex.filename, str):
   	message = f"{message} Filename '{ex.filename}'."
   	tool_error = ToolCallError("permission", message)
   	except FileNotFoundError as ex:
   	tool_error = ToolCallError(
   	"file_not_found",
   	f"File '{ex.filename}' was not found.",
   	)
   	except ToolParsingError as ex:
   	tool_error = ToolCallError("parsing", ex.message)
   	except ToolError as ex:
   	tool_error = ToolCallError("unknown", ex.message)

Note that we try to use known attributes of the exception to provide as detailed an error message as we can.

I've done this; is there a test to which we should add this error case?

[jjallaire-aisi]

I've done this; is there a test to which we should add this error case?

Thanks! No, there aren't currently any tests for tool error dispatching.