update

wenerme · Jun 6, 2024 · 02bfdf3 · 02bfdf3
1 parent 9bcbe04
commit 02bfdf3
Show file tree

Hide file tree

Showing 52 changed files with 874 additions and 294 deletions.
diff --git a/about-me/我说.md b/about-me/我说.md
diff --git a/notes/ai/diffusion/README.md b/notes/ai/diffusion/README.md
@@ -4,6 +4,28 @@ title: Diffusion
 
 # Diffusion
 
-- SD - Stable Diffusion
-- https://cyberes.github.io/stable-diffusion-models/
-- EMA
+Diffusion 模型是一种生成模型，它通过逐步添加噪声到数据中，再通过逆向去噪过程，从纯噪声生成高质量的图像。这个过程类似于将一个清晰的图像逐步变模糊，然后再一步步恢复成清晰图像的过程。
+
+- Forward Diffusion Process - 前向扩散过程
+  - 从数据开始逐步添加噪声，将数据转换为纯噪声
+  - 每一步添加少量噪声，直到数据变得不可辨认
+  - 该过程通常通过一系列预定义的噪声调度（noise schedule）来实现
+- Reverse Diffusion Process - 逆向扩散过程
+  - 学习如何从纯噪声逐步去噪，恢复原始数据
+  - 通过训练模型，使其能够逐步逆转噪声过程，生成逼真的图像
+- 应用
+  - 图像生成
+  - 图像修复
+  - 图像超分辨率
+- 优势
+  - 高质量生成
+  - 稳定训练
+    - 比 GAN 更稳定，不容易出现模式崩溃的问题。
+
+| abbr.  | for                                    | cn             |
+| ------ | -------------------------------------- | -------------- |
+| SD     | Stable Diffusion                       | 稳定扩散       |
+| EMA    | Exponential Moving Average             | 指数移动平均   |
+| VAE    | Variational Autoencoder                | 变分自动编码器 |
+| MSE    | Mean Squared Error                     | 均方误差       |
+| ft-EMA | Fine-Tuning Exponential Moving Average | EMA微调        |
diff --git a/notes/ai/diffusion/diffusion-awesome.md b/notes/ai/diffusion/diffusion-awesome.md
@@ -7,8 +7,16 @@ tags:
 # Stable Diffusion Awesome
 
 - [apple/ml-stable-diffusion](https://github.com/apple/ml-stable-diffusion)
-- https://github.com/CompVis/latent-diffusion
-- https://latent-consistency-models.github.io/
+  - MIT, Python, Swift
+  - Stable Diffusion with Core ML on Apple Silicon
+- [huggingface/diffusers](https://github.com/huggingface/diffusers)
+  - Apache-2.0, Python
+- [CompVis/latent-diffusion](https://github.com/CompVis/latent-diffusion)
+  - MIT
+  - High-Resolution Image Synthesis with Latent Diffusion Models
+  - https://latent-consistency-models.github.io/
+- Fundation
+  - https://github.com/CompVis
 - https://www.fal.ai/models/latent-consistency-sd
   - https://github.com/fal-ai
   - https://huggingface.co/spaces/fal-ai/realtime-stable-diffusion
@@ -59,3 +67,11 @@ tags:
   - https://journeyintoai.com/
   - https://www.midlibrary.io/styles
   - https://www.midlibrary.io/categories/photographers
+- [Stable Diffusion Models](https://cyberes.github.io/stable-diffusion-models/)
+- [openai/guided-diffusion](https://github.com/openai/guided-diffusion)
+
+| en                    | cn       | notes |
+| --------------------- | -------- | ----- |
+| Simple Diffusion      | 简单扩散 |
+| Facilitated Diffusion | 促进扩散 |
+| Active Transport      | 主动运输 |
diff --git a/notes/ai/ml/cvat.md b/notes/ai/ml/cvat.md
@@ -9,26 +9,8 @@ title: CVAT
   - by Intel
   - 支持 OpenCV
   - 支持基于模型的自动化标注
-- Interactors - 用于 Segmentation
-  - Segment Anything Model (SAM)
-  - Deep extreme cut (DEXTR)
-  - Feature backpropagating refinement scheme (f-BRS)
-  - High Resolution Net (HRNet)
-  - Inside-Outside-Guidance (IOG)
-  - Intelligent scissors - OpenCV
-- Detectors
-  - Mask RCNN
-  - Faster RCNN
-  - YOLO v3
-  - Semantic segmentation for ADAS
-  - RetinaNet
-  - Face Detection
-- Trackers
-  - TrackerMIL - OpenCV - https://learnopencv.com/tag/mil/
-  - SiamMask - [foolwood/SiamMask](https://github.com/foolwood/SiamMask)
-  - TransT - Transformer Tracking -  [chenxin-dlut/TransT](https://github.com/chenxin-dlut/TransT)
 - 参考
-  - YoloV8 serverlesss support  [#6471](https://github.com/cvat-ai/cvat/issues/6471)
+  - YoloV8 serverlesss support [#6471](https://github.com/cvat-ai/cvat/issues/6471)
     - 由于 AGPL 原因无法合并 [#6472](https://github.com/cvat-ai/cvat/pull/6472)
   - https://docs.cvat.ai/docs/manual/advanced/ai-tools/
 
@@ -42,18 +24,16 @@ docker compose pull
 # 推荐修改 volumns
 mkdir -p ./data/{db,data,keys,logs,inmem_db,events_db,cache_db}
 
-# http://localhost:8080
-docker compose up
-
-# 自动化标注 - AI Tool
+# http://localhost:8080 cvat
+# http://localhost:8070 nuclio
+# docker compose up
+# 自动化标注 - AI Tool 依赖 nuclio serverless runtime
 # https://docs.cvat.ai/docs/administration/advanced/installation_automatic_annotation/
 # 如果修改了注意添加  --build
-# 使用 nuclio 作为 serverless runtime
 # 部署 nuclio/dashboard
 # 为 server 添加 CVAT_SERVERLESS=1
 # 添加 额外的 host 信息
 docker compose -f docker-compose.yml -f components/serverless/docker-compose.serverless.yml up
-
 ```
 
 ```yaml
@@ -106,13 +86,20 @@ volumes:
 
 ## serverless
 
-- nuctl
-- https://github.com/nuclio/nuclio
-
 ```bash
+# https://github.com/nuclio/nuclio/releases/
+curl -o nuctl -L https://github.com/nuclio/nuclio/releases/download/1.13.3/nuctl-1.13.3-darwin-$(uname -m)
+chmod +x nuctl
+# 假设 $HOME/bin 在 PATH 中
+mv nuctl ~/bin/
+
+# function.yaml
+# 构建过程会访问 github.com dl.fbaipublicfiles.com pip3
+# 不配置代理大多数情况下是构建不成功的
 ./serverless/deploy_cpu.sh serverless/openvino/dextr
 ./serverless/deploy_cpu.sh serverless/openvino/omz/public/yolo-v3-tf
 
+./serverless/deploy_cpu.sh serverless/pytorch/facebookresearch/sam
 
 # GPU
 nuctl deploy --project-name cvat \
@@ -122,10 +109,69 @@ nuctl deploy --project-name cvat \
   --image cvat/tf.matterport.mask_rcnn_gpu \
   --triggers '{"myHttpTrigger": {"maxWorkers": 1}}' \
   --resource-limit nvidia.com/gpu=1
+
+# quay.io/nuclio/uhttpc:0.0.1-arm6
+# quay.io/nuclio/handler-builder-python-onbuild:1.13.0-arm64
+
+# 依赖 gcr
+docker pull alpine:3.17
+docker tag alpine:3.17 gcr.io/iguazio/alpine:3.17
+# mirror
+crane copy gcr.io/kaniko-project/executor:v1.9.0 registry-vpc.cn-hongkong.aliyuncs.com/cmi/kaniko-project_executor:v1.9.0
+docker pull registry.cn-hongkong.aliyuncs.com/cmi/kaniko-project_executor:v1.9.0
+docker tag registry.cn-hongkong.aliyuncs.com/cmi/kaniko-project_executor:v1.9.0 gcr.io/kaniko-project/executor:v1.9.0
 ```
 
+```bash
+nuctl get function
+```
+
+- nuctl
+- https://github.com/nuclio/nuclio
+
+## AI & OpenCV
+
+- Interactors - 用于 Segmentation, 半自动构建 polygon
+  - Segment Anything Model (SAM)
+  - Deep extreme cut (DEXTR)
+  - Feature backpropagating refinement scheme (f-BRS)
+  - High Resolution Net (HRNet)
+  - Inside-Outside-Guidance (IOG)
+  - Intelligent scissors - OpenCV
+- Detectors
+  - Mask RCNN
+  - Faster RCNN
+  - YOLO v3
+  - Semantic segmentation for ADAS
+  - RetinaNet
+    - detectron2
+  - Face Detection
+- Trackers
+  - TrackerMIL - OpenCV - https://learnopencv.com/tag/mil/
+  - SiamMask - [foolwood/SiamMask](https://github.com/foolwood/SiamMask)
+  - TransT - Transformer Tracking - [chenxin-dlut/TransT](https://github.com/chenxin-dlut/TransT)
+
 # FAQ
 
 ## export skip un-anotated frames
 
 - https://github.com/cvat-ai/cvat/issues/1251
+
+## cvat.openvino.base
+
+## status code 503
+
+
+```bash
+# 检查端口是否通
+nuctl get function
+
+# 检查日志
+docker logs -f  nuclio-nuclio-pth-facebookresearch-sam-vit-h
+# 判断容器内端口是否正常
+docker exec -it nuclio-nuclio-pth-facebookresearch-sam-vit-h curl -v http://localhost:8080
+```
+
+- https://github.com/cvat-ai/cvat/issues/6582
+
+## Failed to parse: http://host.docker.internal:None
diff --git a/notes/ai/ml/detectron2.md b/notes/ai/ml/detectron2.md
@@ -0,0 +1,12 @@
+---
+title: detectron2
+---
+
+# detectron2
+
+- [facebookresearch/detectron2](https://github.com/facebookresearch/detectron2)
+  - Apache-2.0, Python
+  - by Facebook AI Research (FAIR), 2019
+  - platform for object detection, segmentation and other visual recognition
+- 参考
+  - [Model Zoo](https://github.com/facebookresearch/detectron2/blob/main/MODEL_ZOO.md)
diff --git a/notes/ai/ml/gan.md b/notes/ai/ml/gan.md
@@ -10,6 +10,9 @@ title: GAN
   - [eriklindernoren/PyTorch-GAN](https://github.com/eriklindernoren/PyTorch-GAN)
   - [eriklindernoren/Keras-GAN](https://github.com/eriklindernoren/Keras-GAN)
   - [nightrome/really-awesome-gan](https://github.com/nightrome/really-awesome-gan)
+  - https://videogigagan.github.io/
+    - [HN](https://news.ycombinator.com/item?id=40130803)
+  - https://pfnet-research.github.io/tgan/
 - Generator - 生成器
   - 负责生成看起来像真实数据的样本
 - Discriminator - 判别器

diff --git a/notes/ai/ml/ml-awesome.md b/notes/ai/ml/ml-awesome.md
@@ -6,18 +6,28 @@ tags:
 
 # Machine Learning Awesome
 
-- [GPT Awesome](./gpt-awesome.md)
-- [Diffusion Awesome](../diffusion/diffusion-awesome.md)
-- [OCR Awesome](./ocr-awesome.md)
+- [LLM](../llm/README.md)
+  - [LLM Awesome](../llm/llm-awesome.md)
+- [Diffusion](../diffusion/README.md)
+  - [Diffusion Awesome](../diffusion/diffusion-awesome.md)
+- [NLP](../nlp/README.md)
+  - [NLP Awesome](../nlp/nlp-awesome.md)
 - Framework
-  - PyTorch
+  - [PyTorch](./pytorch/README.md)
+    - by Meta
     - [tinygrad/tinygrad](https://github.com/tinygrad/tinygrad)
     - [minitorch/minitorch](https://github.com/minitorch/minitorch)
     - micrograd
-  - JAX
+  - [Lightning-AI/pytorch-lightning](https://github.com/Lightning-AI/pytorch-lightning)
+    - Apache-2.0, Python
+  - [JAX](./jax.md)
+    - by Google
     - [google/maxtext](https://github.com/google/maxtext)
       - simple, performant and scalable Jax LLM
   - [apple/corenet](https://github.com/apple/corenet)
+  - [Paddle](./paddle.md)
+    - by 百度
+  - [OpenCV](./opencv/README.md)
   - Tensorflow
   - Keras
   - AutoKeras
@@ -55,6 +65,49 @@ tags:
   - [Label Studio](./label-studio.md)
   - [CVHub520/X-AnyLabeling](./x-anylabeling.md)
     - GPLv3, Python
+- Deploy/Inference/Serving/Mobile/Optimization
+  - [openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino)
+    - Apache-2.0, C++, Python
+    - optimizing, deploying AI inference
+    - by Intel
+  - [hpcaitech/ColossalAI](https://github.com/hpcaitech/ColossalAI)
+    - Apache-2.0, Python
+  - [microsoft/DeepSpeed](https://github.com/microsoft/DeepSpeed)
+    - Apache-2.0, Python, C++
+    - optimization library that makes distributed training and inference easy, efficient, and effective
+  - [Tencent/ncnn](https://github.com/Tencent/ncnn)
+    - BSD-3, C++, C
+    - neural network inference framework optimized for the mobile platform
+  - ~~[johnolafenwa/deepstack](https://github.com/johnolafenwa/deepstack)~~
+    - Apache-2.0, Go, Python
+    - Cross Platform AI Engine for Edge Devices
+- MLOps
+  - [wandb](https://github.com/wandb/wandb)
+    - MIT, Python, Go
+- VToonify
+- music
+  - [AI-Guru/music-generation-research](https://github.com/AI-Guru/music-generation-research)
+- upscale
+  - [Araxeus/PNG-Upscale](https://github.com/Araxeus/PNG-Upscale)
+    - MIT, Java
+  - [IBM/MAX-Image-Resolution-Enhancer](https://github.com/IBM/MAX-Image-Resolution-Enhancer)
+    - Apache-2.0, Python
+    - Docker
+  - [upscayl/upscayl](https://github.com/upscayl/upscayl)
+    - AGPL-3.0
+    - 需要 GPU
+    - App 方式
+    - Real-ESRGAN
+- image
+  - https://huggingface.co/datasets/vivym/midjourney-messages
+- video
+  - [blakeblackshear/frigate](https://github.com/blakeblackshear/frigate)
+    - MIT, Python, TS
+    - NVR with realtime local object detection for IP cameras
+- tts
+  - [yl4579/StyleTTS2](https://github.com/yl4579/StyleTTS2)
+    - MIT, Python
+    - [HN](https://news.ycombinator.com/item?id=38335255)
 - RT-DETR - Real-Time Detection Transformer
   - by Baidu
 - SAM/Segment Anything Model
@@ -80,9 +133,14 @@ tags:
 - Foundation
   - https://lfaidata.foundation/
   - https://ai.meta.com/
+  - [facebookresearch](https://github.com/facebookresearch)
   - https://ai.google/
   - https://labs.google/
   - https://ai.tencent.com/
+  - [NVlabs](https://github.com/NVlabs)
+    - NVIDIA Research
+  - [THUDM](https://github.com/THUDM)
+    - 清华 KEG & 数据挖掘
 - [PKU-YuanGroup/Video-LLaVA](https://github.com/PKU-YuanGroup/Video-LLaVA)
 - [GokuMohandas/MadeWithML](https://github.com/GokuMohandas/MadeWithML)
 - [josephmisiti/awesome-machine-learning](https://github.com/josephmisiti/awesome-machine-learning)
@@ -234,37 +292,6 @@ tags:
   - [HN](https://news.ycombinator.com/item?id=33027376)
 - [Adventure game graphics with DALL-E 2](https://hpjansson.org/blag/2022/08/16/adventure-game-graphics-with-dall-e-2/)
   - [HN](https://news.ycombinator.com/item?id=32490455)
-- VToonify
-- orgs
-  - [NVlabs](https://github.com/NVlabs)
-  - [facebookresearch](https://github.com/facebookresearch)
-  - [THUDM](https://github.com/THUDM)
-    - 清华 KEG & 数据挖掘
-- music
-  - [AI-Guru/music-generation-research](https://github.com/AI-Guru/music-generation-research)
-- serving
-  - [johnolafenwa/deepstack](https://github.com/johnolafenwa/deepstack)
-- upscale
-  - [Araxeus/PNG-Upscale](https://github.com/Araxeus/PNG-Upscale)
-    - MIT, Java
-  - [IBM/MAX-Image-Resolution-Enhancer](https://github.com/IBM/MAX-Image-Resolution-Enhancer)
-    - Apache-2.0, Python
-    - Docker
-  - [upscayl/upscayl](https://github.com/upscayl/upscayl)
-    - AGPL-3.0
-    - 需要 GPU
-    - App 方式
-    - Real-ESRGAN
-- image
-  - https://huggingface.co/datasets/vivym/midjourney-messages
-- video
-  - [blakeblackshear/frigate](https://github.com/blakeblackshear/frigate)
-    - MIT, Python, TS
-    - NVR with realtime local object detection for IP cameras
-- tts
-  - [yl4579/StyleTTS2](https://github.com/yl4579/StyleTTS2)
-    - MIT, Python
-    - [HN](https://news.ycombinator.com/item?id=38335255)
 
 ---
 
@@ -495,17 +522,6 @@ tags:
   - 工作已经开始转移到 Kaldi, Vosk
   - [cmusphinx/pocketsphinx](https://github.com/cmusphinx/pocketsphinx)
 
-**术语**
-
-| abbr | mean                          | desc |
-| ---- | ----------------------------- | ---- |
-| ASR  | Automatic Speech Recognition  |
-| TTS  | Text-to-speech                |
-| SE   | Speech enhancement/separation |
-| ST   | Speech Translation            |
-| MT   | Machine Translation           |
-| VC   | Voice conversion              |
-
 ## Hardware Platform
 
 - RTX