llm

redhat/notes/2023/2023.11.rhel.gpu.llm.md

@@ -0,0 +1,51 @@
+# 在 RHEL9 上训练 LLM 并实现个人知识库 （RAG）
+
+随着chatgpt出圈，大预言模型火遍全球。那么在红帽的平台上，如何训练/微调一个大预言模型，如何使用大预言模型运行一个应用？
+
+现在，我们就在红帽的RHEL操作系统上，来一步一步的从零开始，下载一个大预言模型，部署一个应用，并且微调这个大预言模型。在这个过程中，体验大语言模型的魅力和给我们带来的挑战。
+
+由于大语言模型的火爆，相关的开源项目欣欣向荣，我们主要会使用2个开源项目，来逐步完成我们的实验。
+
+这里是实验架构图：
+
+![](imgs/2023-11-27-22-59-47.png)
+
+# 驱动安装
+
+我们的实验环境是一台rhel9主机，开始实验的第一件事情，就是按照nvidia驱动
+
+```bash
+
+grubby --update-kernel=ALL --args="amd_iommu=on iommu=on modprobe.blacklist=nouveau"
+
+grub2-mkconfig -o /etc/grub2.cfg
+
+dnf update -y
+
+reboot
+
+dnf groupinstall -y 'development'
+
+dnf groupinstall -y 'server with gui'
+
+
+# or using rpm network
+# https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Rocky&target_version=9&target_type=rpm_network
+dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
+dnf clean all
+dnf -y module install nvidia-driver:latest-dkms
+dnf -y install cuda
+
+# NCCL
+# https://developer.nvidia.com/nccl
+dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo
+
+dnf install -y libnccl libnccl-devel libnccl-static
+
+
+```
+
+# 下载模型
+
+大语言模型的数据文件非常大，我们需要提前下载，不然应用运行的时候，会自动下载，为了方便管理，我们先手动下载下来
+

redhat/ocp4/4.13/4.13.gpu.llm.md

@@ -1169,7 +1169,8 @@ then, you can see the webpage:
 ![](imgs/2023-11-15-00-07-42.png)
 
 
-## Caikit-TGIS-Serving
+
+## Caikit-TGIS-pgvector
 
 - https://github.com/opendatahub-io/caikit-tgis-serving
 
@@ -1823,33 +1824,20 @@ oc delete ns ${TEST_NS} ${MINIO_NS}
 
 - https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-Caikit-Prompt-memory.ipynb
 
+### try using grpc, pgvector, create index
 
-
-### try using grpc, langchain
-
-- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-Caikit-Basic.ipynb
-
-
-```bash
-
-
-```
-
-![](imgs/2023-11-27-17-53-24.png)
-
-回答效果并不好，因为我们的模型太小了，也正常。
-
-### try using grpc, pgvector, langchain
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-PgVector-Ingest.ipynb
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-PgVector-Query.ipynb
 
 ```bash
 
 
 
 ```
 
-### try using grpc, rag, langchain
+### try using grpc, rag, pgvector, langchain
 
-- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/RAG_with_sources_Langchain-Caikit.ipynb
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/RAG_with_sources_Langchain-Caikit-wzh-pgvector.ipynb
 
 ```bash
 
@@ -1878,6 +1866,28 @@ oc delete ns ${TEST_NS} ${MINIO_NS}
 
 ![](imgs/2023-11-27-18-51-40.png)
 
+
+## Chatbot+HFTGI+pgvector
+
+
+### try HFTGI basic
+
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-HFTGI-Basic.ipynb
+
+### try HFTGI memory
+
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/Langchain-HFTGI-Prompt-memory.ipynb
+
+### try HFTGI rag
+
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/RAG_with_sources_Langchain-HFTGI.ipynb
+
+### try HFTGI pgvector, rag
+
+- https://github.com/wangzheng422/llm-on-openshift/blob/wzh/examples/notebooks/langchain/RAG_with_sources_Langchain-Caikit-wzh-pgvector.ipynb
+
+
+
 # end
 
 ![](imgs/2023-11-09-14-39-44.png)
@@ -2017,6 +2027,16 @@ grpcurl -insecure -d '{"text": "At what temperature does liquid Nitrogen boil?"}
 
 
 
+```
+
+# label project
+
+```bash
+
+oc label ns kserve-demo modelmesh-enabled=true
+
+oc label ns kserve-demo opendatahub.io/dashboard=true
+
 ```
 
 # final