Infra: introduce dev-container with debugging support and refactor gr…

…ader (#22)

* chore: update github badges

Signed-off-by: Yiyang Wu <[email protected]>

* CI: use self-maintained preprocessors

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add devcontainer support

Signed-off-by: Yiyang Wu <[email protected]>

* refactor: refactor variable name and scripts

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add py-based scores.json for grader

Signed-off-by: Yiyang Wu <[email protected]>

* infra: refactor out chbuild

Signed-off-by: Yiyang Wu <[email protected]>

* vcs: remove .vscode from .gitignore

Signed-off-by: Yiyang Wu <[email protected]>

* docs: update Getting-Started.md

Signed-off-by: Yiyang Wu <[email protected]>

* docs: update Contribute.md

Signed-off-by: Yiyang Wu <[email protected]>

* infra: introduce gdb 13.2 && qemu 9.1.0

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add chcore_target_precompile

Include a new macro to select the object file
to choose to use whether debug mode or release mode

Signed-off-by: Yiyang Wu <[email protected]>

* fix(lab1): fix linker to include debug symbols

linker.tpl.ld does not include symbols for init objects. This commits fix it.

* lab1: use split-mode precompiled objects

Signed-off-by: Yiyang Wu <[email protected]>

* lab1: move gdbinit

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add clangd and cppdbg support

Signed-off-by: Yiyang Wu <[email protected]>

* lab1: fix procmgr binary

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add additional comment to capturer.py

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add qemu auto launch task

Signed-off-by: Yiyang Wu <[email protected]>

* infra: add arm highlight && hexeditor

Signed-off-by: Yiyang Wu <[email protected]>

* docs: add debugging.md

Signed-off-by: Yiyang Wu <[email protected]>

* docs(lab1): fix typo

Signed-off-by: Yiyang Wu <[email protected]>

* docs(lab0): fix fense

Signed-off-by: Yiyang Wu <[email protected]>

* docs: rearrange SUMMARY.md

Signed-off-by: Yiyang Wu <[email protected]>

* scripts: fix qemu and grade recipes

Signed-off-by: Yiyang Wu <[email protected]>

* scripts: fix gendeps.sh for customized ubuntu

Signed-off-by: Yiyang Wu <[email protected]>

* docs: enhance linker.md

Signed-off-by: Yiyang Wu <[email protected]>

* docs: update Contribute.md

Signed-off-by: Yiyang Wu <[email protected]>

* lab1: remove LICENSE

Signed-off-by: Yiyang Wu <[email protected]>

* docs: add Contribute lab assignment

Signed-off-by: Yiyang Wu <[email protected]>

* docs: fix mdbook guide

Signed-off-by: Yiyang Wu <[email protected]>

* docs(lab1): add RTFSC.md

Signed-off-by: Yiyang Wu <[email protected]>

---------

Signed-off-by: Yiyang Wu <[email protected]>

.devcontainer/Dockerfile

@@ -0,0 +1,69 @@
+FROM ipads/chcore_builder:v1.9.0
+
+ARG USERNAME=stu
+ARG USER_UID=1000
+ARG USER_GID=$USER_UID
+
+RUN sed -i "s/archive.ubuntu.com/mirrors.tuna.tsinghua.edu.cn/g" /etc/apt/sources.list
+RUN apt-get update && apt-get upgrade -y
+RUN apt-get install -y sudo \
+					binutils-dev \
+					libgmp-dev \
+					libmpfr-dev \
+					curl \
+					meson \
+					python3 \
+					python3-pip \
+					python3-psutil \
+					bear \
+					cmake \
+					git \
+					pkgconf \
+					clangd
+
+# We need to install a latest gdb to support cppdbg.
+RUN curl -SLO https://ftp.gnu.org/gnu/gdb/gdb-13.2.tar.xz && \
+	tar -xf gdb-13.2.tar.xz && \
+	cd gdb-13.2 && \
+	./configure --prefix=/usr/local \
+				--enable-targets=aarch64-linux-gnu,x86_64-linux-gnu \
+				--enable-tui \
+				--disable-sim \
+				&& \
+	make -j$(nproc) && \
+	make install && \
+	cd .. && \
+	rm -rf gdb-13.2 gdb-13.2.tar.xz
+
+# install tomli for .toml file required by qemu 9.1.x
+RUN pip install tomli
+
+# we need to install >=glib-2.66.8 to support qemu 9.1.x
+RUN curl -SLO https://download.gnome.org/sources/glib/2.66/glib-2.66.8.tar.xz && \
+	tar -xvf glib-2.66.8.tar.xz && \
+	cd  glib-2.66.8 && \
+	meson build --prefix /usr/local && \
+	ninja -C build && \
+	ninja -C build install && \
+	cd .. && \
+	rm -rf glib-2.66.8 glib-2.66.8.tar.xz
+
+# install latest supported qemu supported by ubuntu-20.04.
+RUN curl -SLO https://download.qemu.org/qemu-9.1.0.tar.xz && \
+	tar -xf qemu-9.1.0.tar.xz && \
+	cd qemu-9.1.0 && \
+	./configure --prefix=/usr/local \
+				--target-list=aarch64-softmmu,aarch64-linux-user && \
+	make -j$(nproc) && \
+	make install && \
+	cd .. && \
+	rm -rf qemu-9.1.0 qemu-9.1.0.tar.xz
+
+RUN apt-get install -y libglib2.0-dev
+RUN apt-get clean
+
+RUN useradd -m -u $USER_UID -U -s /bin/bash stu
+RUN usermod -aG sudo stu
+RUN passwd -d stu
+USER $USERNAME
+CMD ["/bin/bash"]

.devcontainer/devcontainer.json

@@ -0,0 +1,18 @@
+{
+	"name": "OS-Course-Lab-Dev",
+	"build": { "dockerfile": "Dockerfile" },
+	"remoteUser": "stu",
+	"updateRemoteUserUID": true,
+	"runArgs": ["--hostname", "Chcore"],
+	"customizations": {
+		"vscode": {
+		"extensions": [
+			"ms-vscode.cpptools",
+			"ms-vscode.hexeditor",
+			"llvm-vs-code-extensions.vscode-clangd",
+			"ms-vscode.makefile-tools",
+			"dan-c-underwood.arm"
+		]
+	}
+  }
+}

.github/workflows/github-pages.yml

@@ -1,5 +1,4 @@
 name: GitHub Pages
-
 on:
   push:
     sources:
@@ -26,7 +25,7 @@ jobs:
         run: |
           mkdir mdbook
           tag=$(curl 'https://api.github.com/repos/toolmanp/rs-mdbook-callouts/releases/latest' | jq -r '.tag_name')
-          url="https://github.com/toolmanp/rs-mdbook-callouts/releases/download/${tag}/mdbook-callouts-x86_64-unknown-linux-gnu.tar.gz"
+          url="https://github.com/toolmanp/rs-mdbook-callouts/releases/download/${tag}/mdbook-callouts-${tag}-x86_64-unknown-linux-gnu.tar.gz"
           curl -sSL $url | tar -xz --directory=./mdbook
           tag=$(curl 'https://api.github.com/repos/rust-lang/mdbook/releases/latest' | jq -r '.tag_name')
           url="https://github.com/rust-lang/mdbook/releases/download/${tag}/mdbook-${tag}-x86_64-unknown-linux-gnu.tar.gz"
@@ -37,8 +36,8 @@ jobs:
           tag=$(curl 'https://api.github.com/repos/badboy/mdbook-last-changed/releases/latest' | jq -r '.tag_name')
           url="https://github.com/badboy/mdbook-last-changed/releases/download/${tag}/mdbook-last-changed-${tag}-x86_64-unknown-linux-gnu.tar.gz"
           curl -sSL $url | tar -xz --directory=./mdbook
-          tag=$(curl 'https://api.github.com/repos/badboy/mdbook-mermaid/releases/latest' | jq -r '.tag_name')
-          url="https://github.com/badboy/mdbook-mermaid/releases/download/${tag}/mdbook-mermaid-${tag}-x86_64-unknown-linux-gnu.tar.gz"
+          tag=$(curl 'https://api.github.com/repos/toolmanp/mdbook-mermaid/releases/latest' | jq -r '.tag_name')
+          url="https://github.com/toolmanp/mdbook-mermaid/releases/download/${tag}/mdbook-mermaid-${tag}-x86_64-unknown-linux-gnu.tar.gz"
           curl -sSL $url | tar -xz --directory=./mdbook
           echo `pwd`/mdbook >> $GITHUB_PATH
       - name: Setup Pages

.gitignore

@@ -10,7 +10,6 @@ cscope.*
 !/**/firmware/*.elf
 
 build
-.vscode
 simulate.sh
 debug.sh
 .config

Lab0/Makefile

@@ -1,11 +1,9 @@
 LAB := 0
+TIMEOUT := 10
 
-PROJECT := $(CURDIR)/..
-GRADER := $(PROJECT)/Scripts/grader.sh
+include $(CURDIR)/../Scripts/lab.mk
 
-export PROJECT CURDIR LAB
-
-include $(PROJECT)/Scripts/lab.mk
+export LABROOT LABDIR LAB TIMEOUT
 
 bomb: student-number.txt
 	@echo "  GEN bomb"
@@ -17,19 +15,19 @@ clean:
 
 ## execute bomb
 qemu:
-	@qemu-aarch64 bomb
+	@$(QEMU) bomb
 
 qemu-gdb:
-	@qemu-aarch64 -g 1234 bomb
+	@$(QEMU) -g 1234 bomb
 
 qemu-grade:
-	@qemu-aarch64 bomb < ans.txt
+	@$(QEMU) bomb < ans.txt
 
 gdb:
 	$(GDB) -ex "set architecture aarch64" -ex "target remote localhost:1234" -ex "add-symbol-file bomb"
 
 grade:
-	@$(GRADER)
+	@$(GRADER) -t $(TIMEOUT) -f $(CURDIR)/scores.json make qemu-grade
 
 submit:
 	@echo "  SUBMIT lab0"

Lab0/scores.json

@@ -0,0 +1,7 @@
+[
+  {
+    "capture": "Congrats! You have defused all phases!",
+    "msg": "Bomb Defused",
+    "proposed": 100
+  }
+]

Lab0/scripts/generate_bomb.sh

@@ -1,17 +1,17 @@
 #!/bin/bash
 
-if [[ -z $PROJECT ]]; then
-    echo "Please set the PROJECT environment variable to the root directory of your project. (Makefile)"
+if [[ -z $LABROOT ]]; then
+    echo "Please set the LABROOT environment variable to the root directory of your project. (Makefile)"
     exit 1
 fi
 
-if [[ ! -d "${CURDIR}/warehouse" ]]; then
+if [[ ! -d "${LABDIR}/warehouse" ]]; then
     echo "Please run under the root directory of this lab!"
     exit 1
 fi
 
 total_bombs=30
-student=$(cat ${CURDIR}/student-number.txt)
+student=$(cat ${LABDIR}/student-number.txt)
 
 if [[ -z $student ]]; then
     echo "Please enter your student number in student-number.txt at first! Otherwise you would recieve 0 POINTS!"
@@ -20,4 +20,4 @@ fi
 
 bomb_id=$(($student % $total_bombs + 1))
 
-cp ${CURDIR}/warehouse/bomb-${bomb_id} ${CURDIR}/bomb
+cp ${LABDIR}/warehouse/bomb-${bomb_id} ${LABDIR}/bomb

Lab1/.clangd

@@ -0,0 +1,2 @@
+CompileFlags:
+  CompilationDatabasePath: build/kernel/

Lab1/.vscode/launch.json

@@ -0,0 +1,29 @@
+{
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "QEMU (cppdbg)",
+            "type": "cppdbg",
+            "request": "launch",
+            "program": "${workspaceFolder}/build/kernel.img",
+            "args": [],
+            "cwd": "${workspaceFolder}",
+            "environment": [],
+            "externalConsole": false,
+            "stopAtEntry": true,
+            "stopAtConnect": true,
+            "setupCommands": [
+                {
+                    "description": "Enable pretty-printing for gdb",
+                    "text": "-enable-pretty-printing",
+                    "ignoreFailures": true
+                },
+            ],
+            "MIMode": "gdb",
+            "targetArchitecture": "arm64",
+            "miDebuggerServerAddress": "localhost:1234",
+            "miDebuggerPath": "/usr/local/bin/gdb",
+            "preLaunchTask": "Setup QEMU"
+        }
+    ]
+}

Lab1/.vscode/settings.json

@@ -0,0 +1,6 @@
+{
+    "clangd.arguments": [
+        "--compile-commands-dir=build/kernel"
+    ],
+    "debug.allowBreakpointsEverywhere": true
+}

Lab1/.vscode/tasks.json

@@ -0,0 +1,21 @@
+{
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "label": "Setup QEMU",
+            "type": "shell",
+            "isBackground": true,
+            "command": "make qemu-gdb",
+            "problemMatcher": {
+                "pattern": {
+                    "regexp": "^\\[QEMU\\] Waiting for GDB Connection"
+                },
+                "background": {
+                    "activeOnStart": true,
+                    "beginsPattern": "^\\[QEMU\\] Waiting for GDB Connection",
+                    "endsPattern": "^\\[QEMU\\] Waiting for GDB Connection",
+                }
+            }
+        }
+    ]
+}