update

wenerme · Jul 2, 2024 · bfd33c3 · bfd33c3
1 parent 8f8c6f8
commit bfd33c3
Show file tree

Hide file tree

Showing 43 changed files with 763 additions and 211 deletions.
diff --git a/README.md b/README.md
@@ -23,10 +23,12 @@
 - [Auth 服务](https://www.wener.tech/notes/service/auth)
 - [Golang](https://www.wener.tech/notes/languages/go)
 - [Java](https://www.wener.tech/notes/java)
+- [Web开发](https://www.wener.tech/notes/web) - React, NodeJS
+- [VoIP](https://www.wener.tech/notes/voip) - asterisk
+- [运维部署](https://www.wener.tech/notes/devops)
 - [Kubernetes](https://www.wener.tech/notes/devops/kubernetes)
-- [VoIP](https://www.wener.tech/notes/voip)
-- [纪念那些在 BBK 的时光](https://wener.me/story/bbk-memory)
 - [虚拟化](https://www.wener.tech/notes/os/virt)
+- [纪念那些在 BBK 的时光](https://wener.me/story/bbk-memory)
 
 ## Projects Working On
 

diff --git a/notes/ai/ml/cvat.md b/notes/ai/ml/cvat.md
@@ -13,6 +13,8 @@ title: CVAT
   - YoloV8 serverlesss support [#6471](https://github.com/cvat-ai/cvat/issues/6471)
     - 由于 AGPL 原因无法合并 [#6472](https://github.com/cvat-ai/cvat/pull/6472)
   - https://docs.cvat.ai/docs/manual/advanced/ai-tools/
+  - https://github.com/cvat-ai/cvat/issues/5686
+    - Tracker 只能单帧运行
 
 ```bash
 # https://github.com/cvat-ai/cvat/blob/develop/docker-compose.yml
@@ -89,7 +91,9 @@ volumes:
 ```bash
 # https://github.com/nuclio/nuclio/releases/
 curl -o nuctl -L https://github.com/nuclio/nuclio/releases/download/1.13.3/nuctl-1.13.3-darwin-$(uname -m)
+# curl -o nuctl -L https://github.com/nuclio/nuclio/releases/download/1.13.3/nuctl-1.13.3-linux-$(dpkg --print-architecture)
 chmod +x nuctl
+# sudo mv nuctl /usr/local/bin/
 # 假设 $HOME/bin 在 PATH 中
 mv nuctl ~/bin/
 
@@ -110,6 +114,8 @@ nuctl deploy --project-name cvat \
   --triggers '{"myHttpTrigger": {"maxWorkers": 1}}' \
   --resource-limit nvidia.com/gpu=1
 
+./serverless/deploy_gpu.sh serverless/pytorch/facebookresearch/sam
+
 # quay.io/nuclio/uhttpc:0.0.1-arm6
 # quay.io/nuclio/handler-builder-python-onbuild:1.13.0-arm64
 
@@ -140,10 +146,25 @@ nuctl get function
 
 - https://github.com/cvat-ai/cvat/issues/6714
 - [Serverless tutorial](https://docs.cvat.ai/docs/manual/advanced/serverless-tutorial/)
+- 返回结果 https://github.com/cvat-ai/cvat/issues/6332
 - function.yaml
   - metadata - 提供名字相关信息
+    - type detector, interactor, tracker, reid
   - spec.handler - 定义入口
 
+```yaml
+metadata:
+  name: pth-facebookresearch-detectron2-retinanet-r101
+  namespace: cvat
+  annotations:
+    name: RetinaNet R101
+    type: detector
+    spec: |
+      [
+        { "id": 1, "name": "person" }
+      ]
+```
+
 ## AI & OpenCV
 
 - Interactors - 用于 Segmentation, 半自动构建 polygon
@@ -170,6 +191,70 @@ nuctl get function
 
 - https://docs.cvat.ai/docs/manual/advanced/dataset_manifest/
 
+## yolo
+
+```bash
+mkdir -p serverless/pytorch/ultralytics/yolov8
+nano serverless/pytorch/ultralytics/yolov8/nuclio/function-gpu.yaml
+
+./serverless/deploy_gpu.sh serverless/pytorch/ultralytics/yolov8
+```
+
+```py title="main.py"
+import json
+import base64
+from PIL import Image
+import io
+from ultralytics import YOLO
+import supervision as sv
+
+
+def init_context(context):
+    context.logger.info("Init context...  0%")
+    # Change model path for custom model
+    model_path = "/opt/nuclio/model.pt"
+    model = YOLO(model_path)
+    # Read the DL model
+    context.user_data.model = model
+    context.logger.info("Init context...100%")
+
+
+def handler(context, event):
+    context.logger.info("Run yolo-v8 model")
+    data = event.body
+    buf = io.BytesIO(base64.b64decode(data["image"]))
+    threshold = float(data.get("threshold", 0.5))
+    context.user_data.model.conf = threshold
+    image = Image.open(buf)
+    yolo_results = context.user_data.model(image, conf=threshold)[0]
+    labels = yolo_results.names
+    # from_ultralytics in supervision-0.16.0+
+    detections = sv.Detections.from_ultralytics(yolo_results)
+    detections = detections[detections.confidence > threshold]
+    boxes = detections.xyxy
+    conf = detections.confidence
+    class_ids = detections.class_id
+
+    results = []
+    if boxes.shape[0] > 0:
+        for label, score, box in zip(class_ids, conf, boxes):
+            xtl = int(box[0])
+            ytl = int(box[1])
+            xbr = int(box[2])
+            ybr = int(box[3])
+
+            results.append({
+                "confidence": str(score),
+                "label": labels.get(label, "unknown"),
+                "points": [xtl, ytl, xbr, ybr],
+                "type": "rectangle", })
+
+    return context.Response(body=json.dumps(results), headers={},
+                            content_type='application/json', status_code=200)
+```
+
+- 由于 AGPL 原因无法合并 [#6472](https://github.com/cvat-ai/cvat/pull/6472)
+
 # FAQ
 
 ## export skip un-anotated frames
@@ -238,11 +323,9 @@ docker exec -it nuclio-nuclio-pth-facebookresearch-sam-vit-h curl -v http://loca
 ## ffprobe show frame count
 
 ```bash
-
 ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 input.mp4
 ```
 
-
 ## ffmpeg frames
 
 ```bash
@@ -254,3 +337,11 @@ ffmpeg -i v2.mp4 -start_number 0 -b:v 10000k -vsync 0 -an -y -q:v 4 v2/v2-frame_
 ```
 
 - https://github.com/cvat-ai/cvat/issues/818
+
+## hide long label mapper
+
+- 自定义模型的 label 非常多的时候
+
+```js
+$$('.cvat-runner-label-mapper').forEach((v) => (v.style.display = 'none'));
+```
diff --git a/notes/ai/ml/datumaro.md b/notes/ai/ml/datumaro.md
@@ -27,6 +27,24 @@ datum project export -e '/item/annotation' --filter-mode 'i+a' -f --save-images
 datum transform -t random_split ds:yolo -- --subset train:.67 --subset test:.33 # 随机分割数据集
 ```
 
+## Reference
+
+```bash
+# near-duplicated images
+datum prune -m ndr -p </path/to/project/>
+
+datum transform -t ndr -- \
+  --working_subset train
+  --algorithm gradient
+  --num_cut 100
+  --over_sample random
+  --under_sample uniform
+```
+
+- ndr
+  - https://openvinotoolkit.github.io/datumaro/latest/docs/command-reference/context_free/transform.html#ndr
+- https://openvinotoolkit.github.io/datumaro/latest/docs/command-reference/context_free/index.html
+
 # FAQ
 
 ## error: can't find Rust compiler
@@ -42,7 +60,6 @@ brew install rust
 rm -rf $HOME/intel/isip
 ```
 
-
 ## schema
 
 ```ts

diff --git a/notes/ai/ml/gan.md b/notes/ai/ml/gan.md
@@ -27,6 +27,7 @@ title: GAN
   - 有针对性地生成数据
   - 需要额外信息，通常是类标签或其他形式的调整数据。
   - 生成满足特定条件的数据
+- Conditional WGAN
 - SRGAN - Super Resolution GAN
 - BiGAN - Bidirectional GAN
 - CycleGAN
@@ -52,15 +53,21 @@ title: GAN
 - Contextual Attention GAN (CA-GAN)
 - DeepFill v2
 - Attention-based Image Inpainting
-- StyleGAN
+- [NVlabs/stylegan](https://github.com/NVlabs/stylegan)
   - 能够独立地控制图像的不同层次特征
+  - Mapping network
+  - Synthesis network
+  - PG-GAN （progressive growing GAN）
+  - [NVlabs/stylegan2](https://github.com/NVlabs/stylegan2)
+  - -> [NVlabs/stylegan2-ada](https://github.com/NVlabs/stylegan2-ada)
+  - -> [NVlabs/stylegan2-ada-pytorch](https://github.com/NVlabs/stylegan2-ada-pytorch)
   - Gen
     - 新的结构 - 样式网络
     - style mixing - 混合样式
-    - AdaIN -  adaptive instance normalization - 自适应实例归一化
+    - AdaIN - adaptive instance normalization - 自适应实例归一化
     - 更好地控制生成图像的不同层次特征，从而实现更高的图像质量和更细致的控制。
   - Dis
-    - progressive growing  - 渐进式生长
+    - progressive growing - 渐进式生长
 - ACGAN - Auxiliary Classifier GAN - 辅助分类器生成对抗网络
 - UNet Generator GAN
   - https://github.com/boschresearch/unetgan
@@ -69,23 +76,17 @@ title: GAN
   - https://paperswithcode.com/method/patchgan
 - Pix2Pix
   - 需要成对的训练数据
+- ResNet - Residual Networks - 残差网络
+- [orpatashnik/StyleCLIP](https://github.com/orpatashnik/StyleCLIP)
+  - Text-Driven Manipulation of StyleGAN Imagery
 
 ## StyleGAN
 
 - [EvgenyKashin/stylegan2-distillation](https://github.com/EvgenyKashin/stylegan2-distillation)
 
-## CycleGAN
-
-- [junyanz/pytorch-CycleGAN-and-pix2pix](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
-  - Image-to-Image Translation in PyTorch
-
-
-```bash
-git clone https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix.git
-```
-
 ## 参考 {#reference}
 
+- FFHQ - 高清人脸数据集
 - wikipedia [Generative Adversarial Networks](https://en.wikipedia.org/wiki/Generative_adversarial_network)
 - [eriklindernoren/PyTorch-GAN](https://github.com/eriklindernoren/PyTorch-GAN)
 - [eriklindernoren/Keras-GAN](https://github.com/eriklindernoren/Keras-GAN)
@@ -96,6 +97,3 @@ git clone https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix.git
 - [From Scratch - GAN](https://ym2132.github.io/GenerativeAdversarialNetworks_Goodfellow)
   - https://github.com/YM2132/YMPaperImplementations/blob/main/paper_implementations/python_implemenations/GAN_Goodfellow.py
 - https://poloclub.github.io/ganlab/
-- [junyanz/pytorch-CycleGAN-and-pix2pix](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
-  - Image-to-Image Translation in PyTorch
-- [GaParmar/img2img-turbo](https://github.com/GaParmar/img2img-turbo)
diff --git a/notes/ai/ml/img2img-turbo.md b/notes/ai/ml/img2img-turbo.md
@@ -8,9 +8,12 @@ refs:
 
 - [GaParmar/img2img-turbo](https://github.com/GaParmar/img2img-turbo)
   - MIT
+  - One-step image-to-image with Stable Diffusion turbo
+  - sketch2image, day2night, clear2rainy
   - pix2pix-turbo
   - cyclegan-turbo
-  - One-step image-to-image with Stable Diffusion turbo: sketch2image, day2night
+    - 使用 dino 做特征提取
+    - 利用 sd-turbo
 - 模型和训练只能 A <-> B
 - 参考
   - One-Step Image Translation with Text-to-Image Models https://arxiv.org/abs/2403.12036

diff --git a/notes/ai/ml/img2img.md b/notes/ai/ml/img2img.md
@@ -0,0 +1,61 @@
+---
+title: img2img
+---
+
+# img2img
+
+- img2img
+  - changing a given image in a specific or controlled way
+- [Pix2Pix](https://github.com/phillipi/pix2pix)
+  - 成对训练数据
+  - 训练快
+  - 数据集
+    - 需要一定程度的准确性
+- [Pix2PixHD](https://github.com/NVIDIA/pix2pixHD) - 2017
+  - 2048x1024
+  - photorealistic image-to-image
+  - semantic label maps -> photo-realistic images
+  - synthesizing portraits from face label maps
+  - conditional GANs
+- [Vid2Vid](https://github.com/NVIDIA/vid2vid) - 2018
+  - 引入时序判别器来保持时间一致性
+  - https://arxiv.org/abs/1808.06601
+  - [NVlabs/few-shot-vid2vid](https://github.com/NVlabs/few-shot-vid2vid) - 2019
+    - https://arxiv.org/abs/1910.12713
+    - -> [NVlabs/imaginaire](https://github.com/NVlabs/imaginaire)
+- SPADE (Spatially-Adaptive Normalization) - 2019
+- Pix2PixSC (Style-Consistent) (2020)
+- TransGAN (2021)
+- Pix2Pix-Zero (2023)
+- CycleGAN
+  - 无需成对训练数据
+  - 训练慢
+- Pix2Pix with Temporal Consistency
+- RecycleGAN - CycleGAN
+- FOMM (First Order Motion Model)
+- StyleGAN2 with temporal consistency
+- Transformer-based Video Inpainting
+- few-shot learning
+- perceptual loss 感知损失
+  - 改善生成质量
+- total variation loss 总变差损失
+  - 保持图像平滑
+  - 处理模糊和噪声
+- Style Loss 风格损失
+  - 风格迁移
+- Adversarial Loss 对抗损失
+  - 生成器和判别器
+- U-Net
+
+
+---
+
+- [img2img-turbo](./img2img-turbo.md)
+  - One-step image-to-image with Stable Diffusion turbo
+- [junyanz/pytorch-CycleGAN-and-pix2pix](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix)
+  - Image-to-Image Translation in PyTorch
+- [taesungp/contrastive-unpaired-translation](https://github.com/taesungp/contrastive-unpaired-translation)
+  - 2020
+  - https://arxiv.org/abs/2007.15651
+- https://machinelearningmastery.com/a-gentle-introduction-to-pix2pix-generative-adversarial-network/
+
diff --git a/notes/ai/ml/jupyterlab.md b/notes/ai/ml/jupyterlab.md
@@ -0,0 +1,18 @@
+---
+title: Jupyter Lab
+---
+
+# Jupyter Lab
+
+- [jupyterlab/jupyterlab](https://github.com/jupyterlab/jupyterlab)
+  - BSD-3
+  - next-generation user interface for Project Jupyter
+- https://jupyter.org/
+
+
+```bash
+sudo systemctl status jupyter
+
+jupyter lab list
+```
+
diff --git a/notes/ai/ml/ml-awesome.md b/notes/ai/ml/ml-awesome.md
@@ -12,6 +12,8 @@ tags:
   - [Diffusion Awesome](../diffusion/diffusion-awesome.md)
 - [NLP](../nlp/README.md)
   - [NLP Awesome](../nlp/nlp-awesome.md)
+- [GAN](./gan.md)
+- [img2img](./img2img.md)
 - Framework
   - [PyTorch](./pytorch/README.md)
     - by Meta
@@ -343,6 +345,10 @@ tags:
 - https://stanford.edu/~shervine/teaching/
 - DEEP LEARNING COURSE https://fleuret.org/dlc/
   - [HN](https://news.ycombinator.com/item?id=38331200)
+- https://github.com/y33-j3T/Coursera-Deep-Learning
+- 中文
+  - [d2l-ai/d2l-zh](https://github.com/d2l-ai/d2l-zh)
+    - https://zh.d2l.ai
 
 ---
 
@@ -506,10 +512,6 @@ tags:
 - https://modelplace.ai/models
 - [OpenBMB/BMList](https://github.com/OpenBMB/BMList)
 
-## GAN
-
-- [orpatashnik/StyleCLIP](https://github.com/orpatashnik/StyleCLIP)
-
 ## Music
 
 - [microsoft/muzic](https://github.com/microsoft/muzic)