add face landmakrs detection support

docs/doc/en/sidebar.yaml

@@ -61,6 +61,8 @@ items:
         label: YOLO11/v8/v5 object detection
     -   file: vision/face_detection.md
         label: Face and keypoints detection
+    -   file: vision/face_landmarks.md
+        label: Face multi landmarks detection
     -   file: vision/face_recognition.md
         label: Face recognition
     -   file: vision/body_key_points.md
@@ -156,6 +158,8 @@ items:
         label: PINMAP
     -   file: peripheral/gpio.md
         label: GPIO
+    -   file: peripheral/key.md
+        label: Key
     -   file: peripheral/uart.md
         label: UART
     -   file: peripheral/i2c.md

docs/doc/en/vision/face_landmarks.md

@@ -0,0 +1,76 @@
+---
+title: MaixCAM MaixPy Face 478 Keypoints Detection
+update:
+  - date: 2025-01-08
+    version: v1.0
+    author: neucrack
+    content: Add face 478 landmarks detection source code, documentation and demo.
+---
+
+## Introduction
+
+In the previous article [Face Detection](./face_detection.md), we introduced how to detect faces and a few keypoints (e.g., 5 keypoints). This article explains how to detect more keypoints.
+
+More keypoints have a wide range of applications, such as expression detection, facial feature recognition, face swapping, and more.
+
+![face_landmarks](../../assets/face_landmarks.jpg)
+
+<a href="../../assets/maixcam_face_landmarks_full.jpg" target="_blank">full size 478 landmarks image</a>
+
+## Using Face Keypoints Detection in MaixPy
+
+MaixPy integrates MediaPipe's **478 face keypoints** detection, with the results as shown below:
+
+![maixcam_face_landmarks](../../assets/maixcam_face_landmarks_1.jpg)
+
+Video demonstration:
+<video playsinline controls autoplay loop muted preload src="/static/video/maixcam_face_landmarks.mp4" type="video/mp4">
+Classifier Result video
+</video>
+
+### Code Usage
+
+To use this feature, MaixPy version must be >= 4.10.0. Refer to the latest code in [MaixPy/examples](https://github.com/sipeed/MaixPy):
+
+```python
+from maix import camera, display, image, nn, app
+
+detect_conf_th = 0.5
+detect_iou_th = 0.45
+landmarks_conf_th = 0.5
+landmarks_abs = True
+landmarks_rel = False
+max_face_num = 2
+detector = nn.YOLOv8(model="/root/models/yolov8n_face.mud", dual_buff=False)
+landmarks_detector = nn.FaceLandmarks(model="/root/models/face_landmarks.mud")
+
+cam = camera.Camera(detector.input_width(), detector.input_height(), detector.input_format())
+disp = display.Display()
+
+while not app.need_exit():
+    img = cam.read()
+    results = []
+    objs = detector.detect(img, conf_th=detect_conf_th, iou_th=detect_iou_th, sort=1)
+    count = 0
+    for obj in objs:
+        img_std = landmarks_detector.crop_image(img, obj.x, obj.y, obj.w, obj.h, obj.points)
+        if img_std:
+            res = landmarks_detector.detect(img_std, landmarks_conf_th, landmarks_abs, landmarks_rel)
+            if res and res.valid:
+                results.append(res)
+        count += 1
+        if count >= max_face_num:
+            break
+    for res in results:
+        landmarks_detector.draw_face(img, res.points, landmarks_detector.landmarks_num, res.points_z)
+    disp.show(img)
+```
+
+### Explanation of Key Points
+
+- `max_face_num`: Limits the maximum number of faces detected to prevent lag due to too many faces in the frame.
+- `landmarks_abs`: Specifies the coordinates of face keypoints in the original `img`. The `points` variable contains 478 keypoints in the order `x0, y0, x1, y1, ..., x477, y477`.
+- `landmarks_rel`: Outputs coordinates in `img_std` and appends the results to the `points` variable.
+- `points_z`: Represents depth estimation of the keypoints relative to the face's center of gravity. The closer to the camera, the larger the value. If behind the face's center, the value is negative. The values are proportional to the face's width.
+```
+

docs/doc/zh/sidebar.yaml

@@ -61,6 +61,8 @@ items:
         label: YOLO11/v8/v5 物体检测
     -   file: vision/face_detection.md
         label: 人脸及关键点检测
+    -   file: vision/face_landmarks.md
+        label: 人脸多关键点检测
     -   file: vision/face_recognition.md
         label: 人脸识别
     -   file: vision/body_key_points.md
@@ -155,23 +157,25 @@ items:
 -   label: 片上外设
     items:
     -   file: peripheral/pinmap.md
-        label: PINMAP 使用
+        label: PINMAP IO 功能映射
     -   file: peripheral/gpio.md
         label: GPIO 和 点灯
+    -   file: peripheral/key.md
+        label: Key 按键事件
     -   file: peripheral/uart.md
-        label: UART 串口使用
+        label: UART 串口
     -   file: peripheral/i2c.md
-        label: I2C 使用
+        label: I2C
     -   file: peripheral/pwm.md
-        label: PWM 使用
+        label: PWM
     -   file: peripheral/spi.md
-        label: SPI 使用
+        label: SPI
     -   file: peripheral/wdt.md
-        label: WDT 看门狗使用
+        label: WDT 看门狗
     -   file: peripheral/adc.md
-        label: ADC 使用
+        label: ADC 模数转换
     -   file: peripheral/hid.md
-        label: HID Device 使用
+        label: 作为 USB HID 设备
 
 -   label: 片外模块
     items:

docs/doc/zh/vision/face_detection.md

@@ -75,5 +75,9 @@ while not app.need_exit():
 
 你可能注意到这里模型初始化使用了`dual_buff`（默认值就是 `True`），使能 `dual_buff` 参数可以加快运行效率，提高帧率，具体原理和使用注意点见 [dual_buff 介绍](./dual_buff.md)。
 
+## 检测更多关键点
+
+这里提供的人脸关键点检测基本都是一次性检测到几个点，但是缺点是点比较少，要想检测到更多点和更加精准的点，请看[人脸关键点文档](./face_landmarks.md)
+
 
 

docs/doc/zh/vision/face_landmarks.md

@@ -0,0 +1,73 @@
+---
+title: MaixCAM MaixPy 人脸 478 关键点检测
+update:
+  - date: 2025-01-08
+    version: v1.0
+    author: neucrack
+    content: 增加人脸478关键点检测源码、文档、例程
+---
+
+## 简介
+
+前面的文章[人脸检测](./face_detection.md)中介绍了如何检测人脸，以及几个关键点（比如 5 个），本文介绍如何检测更多（478个）关键点。
+
+更多的关键点有更多的用途，比如表情检测，人脸特征识别，换脸等等。
+
+![face_landmarks](../../assets/face_landmarks.jpg)
+
+<a href="../../assets/maixcam_face_landmarks_full.jpg" target="_blank">高清大图， 478个点位置在图中可以看到</a>
+
+## MaixPy 中使用人脸关键点检测
+
+MaixPy 中移植了 MediePipe 的人脸 **478 关键点**检测，效果如下：
+
+![maixcam_face_landmarks](../../assets/maixcam_face_landmarks_1.jpg)
+
+效果视频：
+<video playsinline controls autoplay loop muted preload src="/static/video/maixcam_face_landmarks.mp4" type="video/mp4">
+Classifier Result video
+</video>
+
+
+使用代码（MaixPy 版本必须 >= 4.10.0），最新代码以[MaixPy/examples](https://github.com/sipeed/MaixPy)为准：
+```python
+from maix import camera, display, image, nn, app
+
+detect_conf_th = 0.5
+detect_iou_th = 0.45
+landmarks_conf_th = 0.5
+landmarks_abs = True
+landmarks_rel = False
+max_face_num = 2
+detector = nn.YOLOv8(model="/root/models/yolov8n_face.mud", dual_buff = False)
+landmarks_detector = nn.FaceLandmarks(model="/root/models/face_landmarks.mud")
+
+cam = camera.Camera(detector.input_width(), detector.input_height(), detector.input_format())
+disp = display.Display()
+
+while not app.need_exit():
+    img = cam.read()
+    results = []
+    objs = detector.detect(img, conf_th = detect_conf_th, iou_th = detect_iou_th, sort = 1)
+    count = 0
+    for obj in objs:
+        img_std = landmarks_detector.crop_image(img, obj.x, obj.y, obj.w, obj.h, obj.points)
+        if img_std:
+            res = landmarks_detector.detect(img_std, landmarks_conf_th, landmarks_abs, landmarks_rel)
+            if res and res.valid:
+                results.append(res)
+        count += 1
+        if count >= max_face_num:
+            break
+    for res in results:
+        landmarks_detector.draw_face(img, res.points, landmarks_detector.landmarks_num, res.points_z)
+    disp.show(img)
+```
+
+这里有几个地方解释一下：
+* `max_face_num` 可以限制最多检测的人脸数量，防止画面人脸过多变卡。
+* `landmarks_abs` 指定检测到源图`img`中人脸关键点的坐标，在结果的`points`变量中有`478`个点，以`x0,y0,x1,y1,...,x477,y477`顺序排列。
+* `landmarks_rel` 则输出`img_std`中的坐标，结果追加到`points`变量中。
+* `points_z` 是关键点深度估计，值相对于面部重心，离镜头越近值越大，在面部重心之后则为负值，值与面部宽度成比例。
+
+

examples/vision/ai_vision/nn_face_landmarks.py

@@ -0,0 +1,33 @@
+from maix import camera, display, image, nn, app
+
+
+detect_conf_th = 0.5
+detect_iou_th = 0.45
+landmarks_conf_th = 0.5
+landmarks_abs = True
+landmarks_rel = False
+max_face_num = 2
+detector = nn.YOLOv8(model="/root/models/yolov8n_face.mud", dual_buff = False)
+landmarks_detector = nn.FaceLandmarks(model="/root/models/face_landmarks.mud")
+
+cam = camera.Camera(detector.input_width(), detector.input_height(), detector.input_format())
+disp = display.Display()
+
+while not app.need_exit():
+    img = cam.read()
+    results = []
+    objs = detector.detect(img, conf_th = detect_conf_th, iou_th = detect_iou_th, sort = 1)
+    count = 0
+    for obj in objs:
+        img_std = landmarks_detector.crop_image(img, obj.x, obj.y, obj.w, obj.h, obj.points)
+        if img_std:
+            res = landmarks_detector.detect(img_std, landmarks_conf_th, landmarks_abs, landmarks_rel)
+            if res and res.valid:
+                results.append(res)
+        count += 1
+        if count >= max_face_num:
+            break
+    for res in results:
+        landmarks_detector.draw_face(img, res.points, landmarks_detector.landmarks_num, res.points_z)
+    disp.show(img)
+