Test notebooks with test-eagle (#2463)

There are three changes in this PR 1. (2d2fb9c) **Make notebooks runnable with nb-tester** This commit modifies some of the notebooks to make them executable from start to finish. Most common change is defining `instance` and `backend_name` in a hidden cell and giving example values through a comment, rather than having `backend_name = '<your-backend>'`. 2. (945b8d7) **Add extended strategy and reclassify notebooks** Adds a new "extended" test that runs the notebooks on IBM Quantum's `test-eagle` device. This device accepts jobs and returns random bit strings in a very short time period, which is great for testing notebooks that submit jobs too large for a local simulator. This also allows us to test functions notebooks, which take a backend name (string) of an IBM Quantum device, rather than the backend object itself. 3. (9f3df9e) **Add extended notebook tests action** I originally hoped we could test notebooks with test-eagle in CI, but the jobs can be flaky and take just a little too long to complete. Running these jobs on every commit is too expensive and could block writers. Instead, I've added a new "extended" test that we can trigger manually on branches when we need it. Examples could be dependabot PRs, or PRs to functions notebooks; when the PR is ready, we'll trigger the workflow on the PR branch to check the notebooks run correctly. To trigger the workflow, go to Actions > Test notebook (extended) > Run workflow, then select the PR branch and click "Run workflow". *** Fixes #2240, fixes #2239, and fixes #2198.
Qiskit · Dec 16, 2024 · 6ba4574 · 6ba4574
1 parent 38f3e28
commit 6ba4574
Show file tree

Hide file tree

Showing 10 changed files with 273 additions and 84 deletions.
diff --git a/.github/workflows/notebook-test-extended.yml b/.github/workflows/notebook-test-extended.yml
@@ -0,0 +1,38 @@
+# This code is a Qiskit project.
+#
+# (C) Copyright IBM 2024.
+#
+# This code is licensed under the Apache License, Version 2.0. You may
+# obtain a copy of this license in the LICENSE file in the root directory
+# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
+#
+# Any modifications or derivative works of this code must retain this
+# copyright notice, and modified files need to carry a notice indicating
+# that they have been altered from the originals.
+
+name: Test notebooks that submit jobs
+on:
+  workflow_dispatch:
+jobs:
+  execute:
+    name: Execute notebooks with test-eagle
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Get relevant changed files
+        id: all-changed
+        uses: tj-actions/changed-files@af2816c65436325c50621100d67f6e853cd1b0f1
+        with:
+          files: "{docs/**/*.ipynb,scripts/nb-tester/**/*}"
+          separator: "\n"
+          write_output_files: true
+
+      - name: Setup environment
+        uses: ./.github/actions/set-up-notebook-testing
+        with:
+          ibm-quantum-token: ${{ secrets.IBM_QUANTUM_TEST_TOKEN }}
+          instance: "client-enablement/documentation/qiskit-documenta"
+
+      - name: Execute notebooks
+        run: python scripts/ci/extended-execute-notebooks.py
diff --git a/docs/guides/algorithmiq-tem.ipynb b/docs/guides/algorithmiq-tem.ipynb
@@ -175,7 +175,9 @@
     "service = QiskitRuntimeService()\n",
     "backend_name = service.least_busy(operational=True).name\n",
     "\n",
-    "instance = \"<IQP_HUB/IQP_GROUP/IQP_PROJECT>\"\n",
+    "# Instance is a string of the form \"<IQP_HUB/IQP_GROUP/IQP_PROJECT>\".\n",
+    "# The following line gets the instance of the active QiskitRuntimeService account.\n",
+    "instance = service.active_account()[\"instance\"]\n",
     "\n",
     "pub = (qc, [observable])\n",
     "options = {\"default_precision\": 0.02}\n",

diff --git a/docs/guides/functions.ipynb b/docs/guides/functions.ipynb
@@ -58,37 +58,15 @@
     "\n",
     "    <span id=\"save-account\"></span>**If you are working in a trusted Python environment (such as on a personal laptop or workstation),** use the `save_account()` method to save your credentials locally. ([Skip to the next step](#functions-untrusted) if you are not using a trusted environment, such as a shared or public computer, to authenticate to IBM Quantum Platform.)\n",
     "\n",
-    "    To use `save_account()`, run `python` in your shell to open a REPL (read-eval-print loop), then enter the following:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "24dc565f-bb5d-4164-ba44-fc04c2fcaafd",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "token = \"<your-token>\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "32919d16-4b5e-439a-8cda-33687a3d96b4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from qiskit_ibm_catalog import QiskitFunctionsCatalog\n",
+    "    To use `save_account()`, run `python` in your shell to open a REPL (read-eval-print loop), then enter the following:\n",
     "\n",
-    "QiskitFunctionsCatalog.save_account(token=token)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e458aad5-fa7c-4ba1-8e9e-be770d89a04a",
-   "metadata": {},
-   "source": [
-    "Close out of the REPL with `exit()`. From now on, whenever you need to authenticate to the service, you can load your credentials with `QiskitFunctionsCatalog()`."
+    "    ```python\n",
+    "    from qiskit_ibm_catalog import QiskitFunctionsCatalog\n",
+    "\n",
+    "    QiskitFunctionsCatalog.save_account(token=\"<your-token>\")\n",
+    "    ```\n",
+    "\n",
+    "    Close out of the REPL with `exit()`. From now on, whenever you need to authenticate to the service, you can load your credentials with `QiskitFunctionsCatalog()`."
    ]
   },
   {
@@ -99,6 +77,8 @@
    "outputs": [],
    "source": [
     "# Load saved credentials\n",
+    "from qiskit_ibm_catalog import QiskitFunctionsCatalog\n",
+    "\n",
     "catalog = QiskitFunctionsCatalog()"
    ]
   },
@@ -190,7 +170,31 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": null,
+   "id": "139ac1d5-3727-4f44-a26c-0d8a1d8a58f2",
+   "metadata": {
+    "tags": [
+     "remove-cell"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# This cell is hidden from users\n",
+    "# It gets these details programatically so we can test this notebook\n",
+    "from qiskit_ibm_runtime import QiskitRuntimeService\n",
+    "from qiskit.circuit.random import random_circuit\n",
+    "\n",
+    "service = QiskitRuntimeService()\n",
+    "instance = service.active_account()[\"instance\"]\n",
+    "backend_name = service.least_busy().name\n",
+    "\n",
+    "circuit = random_circuit(num_qubits=2, depth=2, seed=42)\n",
+    "observable = \"Z\" * circuit.num_qubits"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
    "id": "9b9a7a3c-cc98-4c19-93cd-f59793515c70",
    "metadata": {},
    "outputs": [
@@ -203,7 +207,11 @@
     }
    ],
    "source": [
-    "job = ibm_cf.run(instance=..., pubs=[], backend=\"backend_name\")\n",
+    "job = ibm_cf.run(\n",
+    "    pubs=[(circuit, observable)],\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_kyiv\"\n",
+    ")\n",
     "\n",
     "job.job_id"
    ]

diff --git a/docs/guides/ibm-circuit-function.ipynb b/docs/guides/ibm-circuit-function.ipynb
@@ -342,7 +342,11 @@
    "cell_type": "code",
    "execution_count": 8,
    "id": "4070e592",
-   "metadata": {},
+   "metadata": {
+     "tags": [
+       "raises-exception"
+     ]
+   },
    "outputs": [
     {
      "name": "stdout",

diff --git a/docs/guides/q-ctrl-optimization-solver.ipynb b/docs/guides/q-ctrl-optimization-solver.ipynb
@@ -139,11 +139,6 @@
     "# If you have not previously saved your credentials, follow instructions at\n",
     "# https://docs.quantum.ibm.com/guides/setup-channel#iqp\n",
     "# to authenticate with your API token.\n",
-    "hub = \"<YOUR_IQP_HUB>\"\n",
-    "group = \"<YOUR_IQP_GROUP>\"\n",
-    "project = \"<YOUR_IQP_PROJECT>\"\n",
-    "\n",
-    "# Authentication\n",
     "catalog = QiskitFunctionsCatalog()\n",
     "\n",
     "# Access Function\n",
@@ -252,7 +247,26 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": null,
+   "id": "cf53550c",
+   "metadata": {
+    "tags": [
+     "remove-cell"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# This cell is hidden from users\n",
+    "from qiskit_ibm_runtime import QiskitRuntimeService\n",
+    "\n",
+    "service = QiskitRuntimeService()\n",
+    "instance = service.active_account()[\"instance\"]\n",
+    "backend_name = service.least_busy().name"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
    "id": "16c66d64",
    "metadata": {},
    "outputs": [],
@@ -261,7 +275,8 @@
     "maxcut_job = solver.run(\n",
     "    problem=problem_as_str,\n",
     "    problem_type=\"maxcut\",\n",
-    "    instance=hub + \"/\" + group + \"/\" + project,\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_kyiv\"\n",
     ")"
    ]
   },
@@ -476,7 +491,9 @@
    "source": [
     "# Solve the problem\n",
     "mvc_job = solver.run(\n",
-    "    problem=srepr(cost_function), instance=hub + \"/\" + group + \"/\" + project\n",
+    "    problem=srepr(cost_function),\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_kyiv\"\n",
     ")"
    ]
   },

diff --git a/docs/guides/q-ctrl-performance-management.ipynb b/docs/guides/q-ctrl-performance-management.ipynb
@@ -103,11 +103,6 @@
     "# If you have not previously saved your credentials, follow instructions at\n",
     "# https://docs.quantum.ibm.com/guides/functions\n",
     "# to authenticate with your API token.\n",
-    "hub = \"<YOUR_IQP_HUB>\"\n",
-    "group = \"<YOUR_IQP_GROUP>\"\n",
-    "project = \"<YOUR_IQP_PROJECT>\"\n",
-    "\n",
-    "# Authentication\n",
     "catalog = QiskitFunctionsCatalog()\n",
     "\n",
     "# Access Function\n",
@@ -203,22 +198,38 @@
     "Run the circuit and optionally define the backend and number of shots."
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2b892956",
+   "metadata": {
+    "tags": [
+     "remove-cell"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# This cell is hidden from users\n",
+    "from qiskit_ibm_runtime import QiskitRuntimeService\n",
+    "\n",
+    "service = QiskitRuntimeService()\n",
+    "instance = service.active_account()[\"instance\"]\n",
+    "backend_name = service.least_busy().name"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
    "id": "26952e1f",
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Choose a backend or remove this option to default to the least busy device\n",
-    "backend_name = \"<CHOOSE_A_BACKEND>\"\n",
-    "\n",
     "# Run the circuit using the estimator\n",
     "qctrl_estimator_job = perf_mgmt.run(\n",
     "    primitive=\"estimator\",\n",
     "    pubs=estimator_pubs,\n",
-    "    instance=hub + \"/\" + group + \"/\" + project,\n",
-    "    backend_name=backend_name,\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_kyiv\", or omit to default to the least busy device\n",
     ")"
    ]
   },
@@ -447,15 +458,12 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Choose a backend or remove this option to default to the least busy device\n",
-    "backend_name = \"<CHOOSE_A_BACKEND>\"\n",
-    "\n",
     "# Run the circuit using the sampler\n",
     "qctrl_sampler_job = perf_mgmt.run(\n",
     "    primitive=\"sampler\",\n",
     "    pubs=sampler_pubs,\n",
-    "    instance=hub + \"/\" + group + \"/\" + project,\n",
-    "    backend_name=backend_name,\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_kyiv\", or omit to default to the least busy device\n",
     ")"
    ]
   },

diff --git a/docs/guides/qedma-qesem.ipynb b/docs/guides/qedma-qesem.ipynb
@@ -117,6 +117,25 @@
     "To get started, try this basic example of estimating the required QPU time to run the QESEM for a given `pub`:"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2fb4efc4",
+   "metadata": {
+    "tags": [
+     "remove-cell"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# This cell is hidden from users\n",
+    "from qiskit_ibm_runtime import QiskitRuntimeService\n",
+    "\n",
+    "service = QiskitRuntimeService()\n",
+    "instance = service.active_account()[\"instance\"]\n",
+    "backend_name = service.least_busy().name"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -138,12 +157,12 @@
     ")\n",
     "\n",
     "job = qesem_function.run(\n",
-    "    instance=\"hub1/group1/project1\",\n",
     "    pubs=[(circ, [avg_magnetization, other_observable])],\n",
     "    options={\n",
     "        \"estimate_time_only\": True,\n",
     "    },\n",
-    "    backend_name=\"ibm_brisbane\",\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_brisbane\"\n",
     ")"
    ]
   },
@@ -163,9 +182,9 @@
    "outputs": [],
    "source": [
     "job = qesem_function.run(\n",
-    "    instance=\"hub1/group1/project1\",\n",
     "    pubs=[(circ, [avg_magnetization, other_observable])],\n",
-    "    backend_name=\"ibm_brisbane\",\n",
+    "    instance=instance,  # E.g. \"ibm-q/open/main\"\n",
+    "    backend_name=backend_name,  # E.g. \"ibm_brisbane\"\n",
     ")"
    ]
   },