llm

redhat/notes/2023/2023.11.rhel.gpu.llm.md

@@ -42,10 +42,150 @@ dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda
 
 dnf install -y libnccl libnccl-devel libnccl-static
 
+curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
+yum install -y git-lfs
+
+dnf install -y conda
+
 
 ```
 
 # 下载模型
 
 大语言模型的数据文件非常大，我们需要提前下载，不然应用运行的时候，会自动下载，为了方便管理，我们先手动下载下来
 
+```bash
+
+# config python to point to python 3.11
+alternatives --config python
+alternatives --config python3
+
+rm -rf /data/py_env/hg_cli/venv
+mkdir -p /data/py_env/hg_cli/
+
+cd /data/py_env/hg_cli
+python3 -m venv venv
+
+# very important, run every time when using python
+source /data/py_env/hg_cli/venv/bin/activate
+
+python -m pip install --upgrade pip setuptools wheel
+
+pip install --upgrade huggingface_hub
+
+# on helper
+# for chatglm2-6b
+VAR_NAME=THUDM/ChatGLM2-6B
+
+VAR_NAME_FULL=${VAR_NAME//\//-}
+echo $VAR_NAME_FULL
+# THUDM-ChatGLM2-6B
+
+mkdir -p /data01/huggingface/${VAR_VAR_NAME_FULLNAME}
+cd /data01/huggingface/${VAR_NAME_FULL}
+
+while true; do
+    huggingface-cli download --repo-type model --revision main --cache-dir /data01/huggingface/cache --local-dir ./ --local-dir-use-symlinks False --resume-download ${VAR_NAME} 
+    if [ $? -eq 0 ]; then
+        break
+    fi
+    sleep 1  # Optional: waits for 1 second before trying again
+done
+
+################
+# for m3e-large
+
+VAR_NAME=moka-ai/m3e-large
+
+VAR_NAME_FULL=${VAR_NAME//\//-}
+echo $VAR_NAME_FULL
+# moka-ai-m3e-large
+
+mkdir -p /data01/huggingface/${VAR_VAR_NAME_FULLNAME}
+cd /data01/huggingface/${VAR_NAME_FULL}
+
+while true; do
+    huggingface-cli download --repo-type model --revision main --cache-dir /data01/huggingface/cache --local-dir ./ --local-dir-use-symlinks False --resume-download ${VAR_NAME} 
+    if [ $? -eq 0 ]; then
+        break
+    fi
+    sleep 1  # Optional: waits for 1 second before trying again
+done
+
+```
+
+# 运行LLM应用
+
+基于LLM的应用，现在看有3个大的方向，一个是chatgpt这样的智能问答，一个是构建个人知识库（RAG），最后一个是AI Agent (function call)。
+
+由于我们使用的是离线的LLM，泛化能力不够，所以AI Agent相关的功能并不能很好的支撑，那么我们就集中精力，在前个场景。
+
+我们选择了一个开源项目，[Langchain-Chatchat](https://github.com/chatchat-space/Langchain-Chatchat)，类似的开源项目有很多，我们选择这个项目，是因为这个项目文档很丰富，源代码结构简单，支持的应用场景很完整。
+
+```bash
+
+mkdir -p /data/env/
+conda create -p /data/env/chatchat python=3.10
+
+conda init bash
+
+conda activate /data/env/chatchat
+# conda deactivate
+
+pip3 install --upgrade pip
+pip install peft
+
+
+mkdir -p /data/git_env
+cd /data/git_env
+
+git clone https://github.com/chatchat-space/Langchain-Chatchat
+
+export ENV_CWD="/data/git_env/Langchain-Chatchat"
+
+cd ${ENV_CWD}
+pip install -r requirements.txt
+
+
+cd ${ENV_CWD}/configs
+/bin/cp -f model_config.py.example model_config.py
+/bin/cp -f server_config.py.example server_config.py
+/bin/cp -f kb_config.py.exmaple kb_config.py
+/bin/cp -f basic_config.py.example basic_config.py
+/bin/cp -f prompt_config.py.example prompt_config.py
+
+# apply custom config
+/bin/cp -f model_config.multi.py model_config.py
+
+
+# init vector db
+cd ${ENV_CWD}
+# /bin/rm -rf ${ENV_CWD}/info.db
+# /bin/rm -rf ${ENV_CWD}/samples/vector_store
+/bin/rm -rf ${ENV_CWD}/knowledge_base/*/vector_store
+python3 init_database.py --recreate-vs
+
+# startup the UI
+# no proxy to run ...
+unset http_proxy
+unset https_proxy
+unset no_proxy
+
+cd ${ENV_CWD}
+python startup.py -a
+
+# 服务端运行信息：
+#     OpenAI API Server: http://0.0.0.0:20000/v1
+#     Chatchat  API  Server: http://0.0.0.0:7861
+#     Chatchat WEBUI Server: http://0.0.0.0:8501
+#     Collecting usage statistics. To deactivate, set browser.gatherUsageStats to False.
+
+# http://172.21.6.98:8501
+
+```
+
+# 微调LLM
+
+
+# end
+