Built site for gh-pages

UKGovernmentBEIS · Apr 28, 2024 · 44502c4 · 44502c4
1 parent 424792e
commit 44502c4
Show file tree

Hide file tree

Showing 33 changed files with 2,272 additions and 958 deletions.
diff --git a/.nojekyll b/.nojekyll
@@ -1 +1 @@
-101118bc
+c01810e3
diff --git a/Inspect.docx b/Inspect.docx
diff --git a/Inspect.epub b/Inspect.epub
diff --git a/Inspect.pdf b/Inspect.pdf
diff --git a/datasets.html b/datasets.html
diff --git a/eval-logs.html b/eval-logs.html
diff --git a/eval-suites.html b/eval-suites.html
diff --git a/eval-tuning.html b/eval-tuning.html
@@ -191,6 +191,11 @@
   <div class="sidebar-item-container"> 
   <a href="./workflow.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Workflow</span></a>
   </div>
+</li>
+          <li class="sidebar-item">
+  <div class="sidebar-item-container"> 
+  <a href="./log-viewer.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Log Viewer</span></a>
+  </div>
 </li>
           <li class="sidebar-item">
   <div class="sidebar-item-container"> 
@@ -921,6 +926,11 @@ <h3 class="anchored" data-anchor-id="web-requests">Web Requests</h3>
     <li class="nav-item">
     <a class="nav-link" href="https://github.com/UKGovernmentBEIS/inspect_ai">
 <p>Code</p>
+</a>
+  </li>  
+    <li class="nav-item">
+    <a class="nav-link" href="https://github.com/UKGovernmentBEIS/inspect_ai/blob/main/CHANGELOG.md">
+<p>Changelog</p>
 </a>
   </li>  
     <li class="nav-item">

diff --git a/examples.html b/examples.html
diff --git a/images/inspect-view-answers.png b/images/inspect-view-answers.png
diff --git a/images/inspect-view-filter.png b/images/inspect-view-filter.png
diff --git a/images/inspect-view-history.png b/images/inspect-view-history.png
diff --git a/images/inspect-view-home.png b/images/inspect-view-home.png
diff --git a/images/inspect-view-info.png b/images/inspect-view-info.png
diff --git a/images/inspect-view-logging-console.png b/images/inspect-view-logging-console.png
diff --git a/images/inspect-view-logging.png b/images/inspect-view-logging.png
diff --git a/images/inspect-view-main.png b/images/inspect-view-main.png
diff --git a/images/inspect-view-messages.png b/images/inspect-view-messages.png
diff --git a/images/inspect-view-metadata.png b/images/inspect-view-metadata.png
diff --git a/images/inspect-view-scoring.png b/images/inspect-view-scoring.png
diff --git a/images/inspect-view-sort.png b/images/inspect-view-sort.png
diff --git a/index.html b/index.html
@@ -74,6 +74,9 @@
 <script src="site_libs/bootstrap/bootstrap.min.js"></script>
 <link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
 <link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" id="quarto-bootstrap" data-mode="light">
+<script src="site_libs/quarto-contrib/glightbox/glightbox.min.js"></script>
+<link href="site_libs/quarto-contrib/glightbox/glightbox.min.css" rel="stylesheet">
+<link href="site_libs/quarto-contrib/glightbox/lightbox.css" rel="stylesheet">
 <script id="quarto-search-options" type="application/json">{
   "location": "sidebar",
   "copy-button": false,
@@ -191,6 +194,11 @@
   <div class="sidebar-item-container"> 
   <a href="./workflow.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Workflow</span></a>
   </div>
+</li>
+          <li class="sidebar-item">
+  <div class="sidebar-item-container"> 
+  <a href="./log-viewer.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Log Viewer</span></a>
+  </div>
 </li>
           <li class="sidebar-item">
   <div class="sidebar-item-container"> 
@@ -419,6 +427,10 @@ <h2 class="anchored" data-anchor-id="sec-hello-inspect">Hello, Inspect</h2>
 <div class="sourceCode" id="cb8"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="ex">$</span> inspect eval theory_of_mind.py <span class="at">--model</span> openai/gpt-4</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
 <p><img src="images/running-theory.png" class="img-fluid"></p>
 <p>By default, eval logs are written to the <code>./logs</code> sub-directory of the current working directory. When the eval is complete you will find a link to the log at the bottom of the task results summary.</p>
+<p>You can also explore eval results using the Inspect log viewer. Run <code>inspect view</code> to open the viewer (you only need to do this once as the viewer will automatically updated when new evals are run):</p>
+<div class="sourceCode" id="cb9"><pre class="sourceCode bash code-with-copy"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="ex">$</span> inspect view</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
+<p><a href="images/inspect-view-home.png" class="lightbox" data-gallery="quarto-lightbox-gallery-1"><img src="images/inspect-view-home.png" class="border img-fluid"></a></p>
+<p>See the <a href="log-viewer.html">Log Viewer</a> section for additional details on using Inspect View.</p>
 <div class="callout callout-style-simple callout-note">
 <div class="callout-body d-flex">
 <div class="callout-icon-container">
@@ -435,6 +447,7 @@ <h2 class="anchored" data-anchor-id="learning-more">Learning More</h2>
 <p>To get stared with Inspect, we highly recommend you read at least these sections for a high level overview of the system:</p>
 <ul>
 <li><p><a href="workflow.html">Workflow</a> covers the mechanics of running evaluations, including how to create evals in both scripts and notebooks, specifying configuration and options, how to parameterise tasks for different scenarios, and how to work with eval log files.</p></li>
+<li><p><a href="log-viewer.html">Log Viewer</a> goes into more depth on how to use Inspect View to develop and debug evaluations, including how to provide additional log metadata and how to integrate it with Python’s standard logging module.</p></li>
 <li><p><a href="examples.html">Examples</a> provides several complete examples with commentary on the use of various features (as with the above example, they are fairly simplistic for the purposes of illustration). You can also find implementations of a few popular <a href="https://github.com/UKGovernmentBEIS/inspect_ai/tree/main/benchmarks">LLM benchmarks</a> in the Inspect repository.</p></li>
 </ul>
 <p>These sections provide a more in depth treatment of the various components used in evals. Read them as required as you learn to build evaluations.</p>
@@ -910,6 +923,11 @@ <h2 class="anchored" data-anchor-id="learning-more">Learning More</h2>
     <li class="nav-item">
     <a class="nav-link" href="https://github.com/UKGovernmentBEIS/inspect_ai">
 <p>Code</p>
+</a>
+  </li>  
+    <li class="nav-item">
+    <a class="nav-link" href="https://github.com/UKGovernmentBEIS/inspect_ai/blob/main/CHANGELOG.md">
+<p>Changelog</p>
 </a>
   </li>  
     <li class="nav-item">
@@ -942,6 +960,31 @@ <h2 class="anchored" data-anchor-id="learning-more">Learning More</h2>
     </div>
   </div>
 </footer>
+<script>var lightboxQuarto = GLightbox({"closeEffect":"zoom","openEffect":"zoom","descPosition":"bottom","loop":false,"selector":".lightbox"});
+window.onload = () => {
+  lightboxQuarto.on('slide_before_load', (data) => {
+    const { slideIndex, slideNode, slideConfig, player, trigger } = data;
+    const href = trigger.getAttribute('href');
+    if (href !== null) {
+      const imgEl = window.document.querySelector(`a[href="${href}"] img`);
+      if (imgEl !== null) {
+        const srcAttr = imgEl.getAttribute("src");
+        if (srcAttr && srcAttr.startsWith("data:")) {
+          slideConfig.href = srcAttr;
+        }
+      }
+    } 
+  });
+
+  lightboxQuarto.on('slide_after_load', (data) => {
+    const { slideIndex, slideNode, slideConfig, player, trigger } = data;
+    if (window.Quarto?.typesetMath) {
+      window.Quarto.typesetMath(slideNode);
+    }
+  });
+
+};
+          </script>