[Bug] [StartUp] Cannot run proxy mode(siliconflow) on a computer with No NVIDIA graphics AND CUDA toolkit

[su1234121]

Search before asking

• I had searched in the issues and found no similar issues.

Operating system information

Linux

Python version information

=3.11

DB-GPT version

main

Related scenes

• Chat Data
• Chat Excel
• Chat DB
• Chat Knowledge
• Model Management
• Dashboard
• Plugins

Installation Information

• Installation From Source

• Docker Installation

• Docker Compose Installation

• Cluster Installation

• AutoDL Image

• Other

Device information

Device CPU

Models information

Qwen/Qwen2.5-Coder-32B-Instruct
BAAI/bge-large-zh-v1.5
BAAI/bge-reranker-v2-m3

What happened

Starting from version 243e981, there is an error message when starting up. Please refer to #2396 for the error issue, After the update to d5a2a0b, I was able to run the project normally and start it on a computer with NVIDIA graphics card and CUDA driver. The next day, due to debugging needs, I pulled the latest version (3bd75d8) on my laptop which is lack of CUDA driver and graphics card on it, which was 3bd75d8 at the time. I thought it was this update that caused the startup problem to recur but have no idea to resolve it. Even today, it has been updated to 93eb3a7. The laptop has also pulled the latest version, but it still starts with an error. I realized that it may be due to the lack of CUDA driver and graphics card on the machine itself. However, this proxy mode startup method should not use CUDA, so I pulled the latest version again on a machine with graphics card for testing. After checking if it can start, I found that it is possible, so I discovered this issue: it will not start on machines without NVIDIA graphics cards and CUDA ToolKit.
By the way, the errors in related #2404 may be caused by the same reason。

What you expected to happen

Based on my understanding of the project introduction video, it should be possible to start in full proxy mode on machines without CUDA or NVIDIA graphics cards

How to reproduce

just find a laptop with no NVIDIA AND CUDA

Additional context

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[fangyinc]

Hi, @su1234121 , thanks for your feedback.
Thank you very much for your attention to our changes.
In addition, what is the complete commans of your installation dependencies?
By the way, if you want to run Qwen/Qwen2.5-Coder-32B-Instruct using cpu, you can consider using llama.cpp to install

[su1234121]

I have tried this two method.
uv sync --all-packages --extra "base" --extra "proxy_openai" --extra "rag" --extra "storage_chromadb" --extra "dbgpts"
uv sync --all-packages --extra "base" --extra "proxy_openai" --extra "rag" --extra "storage_chromadb" --extra "dbgpts" --extra "hf"

thanks for you suggestion, I just want to use siliconflow to have a quick experiment at the moment.
wish this project can be better!

[fangyinc]

I have tried this two method. uv sync --all-packages --extra "base" --extra "proxy_openai" --extra "rag" --extra "storage_chromadb" --extra "dbgpts" uv sync --all-packages --extra "base" --extra "proxy_openai" --extra "rag" --extra "storage_chromadb" --extra "dbgpts" --extra "hf"

thanks for you suggestion, I just want to use siliconflow to have a quick experiment at the moment. wish this project can be better!

Using only siliconflow OK, I just tested it on a new mac m1 computer. Using only the following command works, so I'm not sure why you get an error.

uv sync --all-packages \
--extra "base" \
--extra "proxy_openai" \
--extra "rag" \
--extra "storage_chromadb" \
--extra "dbgpts"

[su1234121]

I delete the file and re git clone and run it with: uv sync --all-packages --extra "base" --extra "proxy_openai" --extra "rag" --extra "storage_chromadb" --extra "dbgpts"
the still get error:
2025-03-10 21:27:37 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_unified_metadata_db_manager_factory and instance: <dbgpt.storage.metadata.db_factory.UnifiedDBManagerFactory object at 0x7fd4cf99aa50>
2025-03-10 21:27:37 localhost.localdomain dbgpt.datasource.rdbms.base[6384] INFO Closing RDBMS connector resources...
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_thread_pool_default and instance: <dbgpt.util.executor_utils.DefaultExecutorFactory object at 0x7fd49b25f8d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_default_scheduler and instance: <dbgpt_app.initialization.scheduler.DefaultScheduler object at 0x7fd49be31cd0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_model_controller and instance: <dbgpt.model.cluster.controller.controller.ModelControllerAdapter object at 0x7fd4ceaa4690>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_connector_manager and instance: <dbgpt_serve.datasource.manages.connector_manager.ConnectorManager object at 0x7fd49ba5a890>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_plugin_hub and instance: <dbgpt_serve.agent.hub.controller.ModulePlugin object at 0x7fd49b2e80d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_multi_agents and instance: <dbgpt_serve.agent.agents.controller.MultiAgents object at 0x7fd49bca9890>
2025-03-10 21:27:38 localhost.localdomain dbgpt_app.initialization.embedding_component[6384] INFO Register remote RemoteEmbeddingFactory
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name embedding_factory and instance: <dbgpt_app.initialization.embedding_component.RemoteEmbeddingFactory object at 0x7fd49b6c8f90>
2025-03-10 21:27:38 localhost.localdomain dbgpt_app.initialization.embedding_component[6384] INFO Register remote RemoteRerankEmbeddingFactory
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name rerank_embedding_factory and instance: <dbgpt_app.initialization.embedding_component.RemoteRerankEmbeddingFactory object at 0x7fd49b16b050>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_model_cache_manager and instance: <dbgpt.storage.cache.manager.LocalCacheManager object at 0x7fd49b2be490>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_awel_trigger_manager and instance: <dbgpt.core.awel.trigger.trigger_manager.DefaultTriggerManager object at 0x7fd49b808c10>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_awel_dag_manager and instance: <dbgpt.core.awel.dag.dag_manager.DAGManager object at 0x7fd4a0157890>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_resource_manager and instance: <dbgpt.agent.resource.manage.ResourceManager object at 0x7fd49b16e810>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_agent_manager and instance: <dbgpt.agent.core.agent_manage.AgentManager object at 0x7fd49bcabc50>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_app_editor_service and instance: <dbgpt_app.openapi.api_v1.editor.service.EditorService object at 0x7fd49bcc7510>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_prompt and instance: <dbgpt_serve.prompt.serve.Serve object at 0x7fd49b7d1050>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_prompt_service and instance: <dbgpt_serve.prompt.service.service.Service object at 0x7fd49b1f34d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_conversation and instance: <dbgpt_serve.conversation.serve.Serve object at 0x7fd49b508310>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_conversation_service and instance: <dbgpt_serve.conversation.service.service.Service object at 0x7fd49b014a90>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_variables_provider and instance: <dbgpt.core.interface.variables.StorageVariablesProvider object at 0x7fd49b16a7d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_flow and instance: <dbgpt_serve.flow.serve.Serve object at 0x7fd49b0486d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_dbgpts_loader and instance: <dbgpt.util.dbgpts.loader.DBGPTsLoader object at 0x7fd49bb2bc10>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_flow_service and instance: <dbgpt_serve.flow.service.service.Service object at 0x7fd49b685bd0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_flow_variables_service and instance: <dbgpt_serve.flow.service.variables_service.VariablesService object at 0x7fd49b27dd10>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.flow.flows and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinFlowVariablesProvider object at 0x7fd49b64ee90>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.flow.nodes and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinNodeVariablesProvider object at 0x7fd49b0c2bd0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.variables and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinAllVariablesProvider object at 0x7fd49bba14d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.secrets and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinAllSecretVariablesProvider object at 0x7fd49a3f3950>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.model.llms and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinLLMVariablesProvider object at 0x7fd49b64d3d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.model.embeddings and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinEmbeddingsVariablesProvider object at 0x7fd49b2bea10>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.datasources and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinDatasourceVariablesProvider object at 0x7fd49b0cffd0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.agent.agents and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinAgentsVariablesProvider object at 0x7fd49b64e850>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt.core.knowledge_spaces and instance: <dbgpt_serve.flow.api.variables_provider.BuiltinKnowledgeSpacesVariablesProvider object at 0x7fd49bcde650>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_rag and instance: <dbgpt_serve.rag.serve.Serve object at 0x7fd49a5fb890>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_rag_service and instance: <dbgpt_serve.rag.service.service.Service object at 0x7fd49a1164d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_datasource and instance: <dbgpt_serve.datasource.serve.Serve object at 0x7fd49a123410>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_datasource_service and instance: <dbgpt_serve.datasource.service.service.Service object at 0x7fd49a14dfd0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_feedback and instance: <dbgpt_serve.feedback.serve.Serve object at 0x7fd4a028b5d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_feedback_service and instance: <dbgpt_serve.feedback.service.service.Service object at 0x7fd49a1948d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_dbgpts_hub and instance: <dbgpt_serve.dbgpts.hub.serve.Serve object at 0x7fd49a28c0d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_dbgpts_hub_service and instance: <dbgpt_serve.dbgpts.hub.service.service.Service object at 0x7fd49a002750>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_dbgpts_my and instance: <dbgpt_serve.dbgpts.my.serve.Serve object at 0x7fd49b0a3950>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_dbgpts_my_service and instance: <dbgpt_serve.dbgpts.my.service.service.Service object at 0x7fd49a0e8410>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_file and instance: <dbgpt_serve.file.serve.Serve object at 0x7fd49b39b290>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_file_service and instance: <dbgpt_serve.file.service.service.Service object at 0x7fd499f7d010>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_evaluate and instance: <dbgpt_serve.evaluate.serve.Serve object at 0x7fd49a2a1b50>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_evaluate_service and instance: <dbgpt_serve.evaluate.service.service.Service object at 0x7fd499e01a50>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_libro and instance: <dbgpt_serve.libro.serve.Serve object at 0x7fd499e026d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_libro_service and instance: <dbgpt_serve.libro.service.service.Service object at 0x7fd499e440d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_model and instance: <dbgpt_serve.model.serve.Serve object at 0x7fd499f8dd90>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_serve_model_service and instance: <dbgpt_serve.model.service.service.Service object at 0x7fd499eaf950>
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name base_dbgpt_component and instance: <dbgpt.util.code.server.CodeServer object at 0x7fd49a2b81d0>
2025-03-10 21:27:38 localhost.localdomain dbgpt.util._db_migration_utils[6384] WARNING Initialize and upgrade database metadata with alembic, just run this in your development environment, if you deploy this in production environment, please run webserver with --disable_alembic_upgrade(python dbgpt/app/dbgpt_server.py --disable_alembic_upgrade).
we suggest you to use dbgpt db migration to initialize and upgrade database metadata with alembic, your can run dbgpt db migration --help to get more information.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Context impl SQLiteImpl.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Will assume non-transactional DDL.
2025-03-10 21:27:38 localhost.localdomain dbgpt.util._db_migration_utils[6384] INFO Migration versions and their file paths:
========================================Migration versions========================================
3ccc5a916731 (current): New migration (Path: /home/suc/DB-GPT/pilot/meta_data/alembic/versions/3ccc5a916731_new_migration.py)

2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Context impl SQLiteImpl.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Will assume non-transactional DDL.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Context impl SQLiteImpl.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Will assume non-transactional DDL.
2025-03-10 21:27:38 localhost.localdomain dbgpt.util._db_migration_utils[6384] INFO alembic migration current revision: 3ccc5a916731, latest revision: 3ccc5a916731
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Context impl SQLiteImpl.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Will assume non-transactional DDL.
Generating /home/suc/DB-
GPT/pilot/meta_data/alembic/versions/bc249d6acdc4_new_migration.py ... done
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Context impl SQLiteImpl.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Will assume non-transactional DDL.
2025-03-10 21:27:38 localhost.localdomain alembic.runtime.migration[6384] INFO Running upgrade 3ccc5a916731 -> bc249d6acdc4, New migration
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_file_storage_client and instance: <dbgpt.core.interface.file.FileStorageClient object at 0x7fd49a072250>
2025-03-10 21:27:38 localhost.localdomain dbgpt_app.dbgpt_server[6384] INFO Model Unified Deployment Mode, run all services in the same process
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Register WorkerManager dbgpt_worker_manager_factory
2025-03-10 21:27:38 localhost.localdomain dbgpt.component[6384] INFO Register component with name dbgpt_worker_manager_factory and instance: <dbgpt.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x7fd4965de190>
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Worker params:
=========================== ModelWorkerParameters ===========================
host: 0.0.0.0
port: 5670
daemon: False
log:
=========================== LoggingParameters ===========================
level: INFO
file: None

trace: None
worker_type: None
worker_class: None
standalone: True
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20

2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:5670
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.adapter.model_adapter[6384] INFO Current model Qwen/Qwen2.5-Coder-32B-Instruct use new adapter <_DynProxyLLMModelAdapter model_name=Qwen/Qwen2.5-Coder-32B-Instruct model_path=None>
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.default_worker[6384] INFO model_name: Qwen/Qwen2.5-Coder-32B-Instruct, model_path: None, model_param_class: <class 'dbgpt.model.proxy.llms.siliconflow.SiliconFlowDeployModelParameters'>
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Init empty instances list for Qwen/Qwen2.5-Coder-32B-Instruct@llm
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Start local embedding worker with embedding parameters
=========================== ModelWorkerParameters ===========================
host: 0.0.0.0
port: 8001
daemon: False
log:
=========================== LoggingParameters ===========================
level: ${env:DBGPT_LOG_LEVEL:-INFO}
file: None

trace: None
worker_type: text2vec
worker_class: None
standalone: False
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20

2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Init empty instances list for BAAI/bge-large-zh-v1.5@text2vec
2025-03-10 21:27:38 localhost.localdomain dbgpt.util.api_utils[6384] WARNING Health check failed for http://127.0.0.1:5670, error: HTTPConnectionPool(host='127.0.0.1', port=5670): Max retries exceeded with url: /api/health (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd496507710>: Failed to establish a new connection: [Errno 111] Connection refused'))
2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Start local embedding worker with embedding parameters
=========================== ModelWorkerParameters ===========================
host: 0.0.0.0
port: 8001
daemon: False
log:
=========================== LoggingParameters ===========================
level: ${env:DBGPT_LOG_LEVEL:-INFO}
file: None

trace: None
worker_type: reranker
worker_class: None
standalone: False
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20

2025-03-10 21:27:38 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Init empty instances list for BAAI/bge-reranker-v2-m3@reranker
2025-03-10 21:27:39 localhost.localdomain dbgpt.util.dbgpts.loader[6384] INFO Found 0 dbgpts packages from /home/suc/.dbgpts/packages/d54e0b1915bd43cdcefac088d97cdc19
2025-03-10 21:27:39 localhost.localdomain dbgpt_serve.flow.service.compat_service[6384] INFO Found compat file: 0.7.0, use latest: 0.7.0
INFO: Started server process [6384]
INFO: Waiting for application startup.
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/awel_flow_ui_components.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/data_analyst_assistant.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=dbgpt_awel_data_analyst_assistant) from mod <module 'unusual_prefix_3e2b4765871ed1ee7574b5846282012767cb1c6a_data_analyst_assistant' from '/home/suc/DB-GPT/examples/awel/data_analyst_assistant.py'> and model file /home/suc/DB-GPT/examples/awel/data_analyst_assistant.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_chat_dag_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=dbgpt_awel_simple_dag_example) from mod <module 'unusual_prefix_c96190b99d7b688a10f384c2185d3b5251c7f742_simple_chat_dag_example' from '/home/suc/DB-GPT/examples/awel/simple_chat_dag_example.py'> and model file /home/suc/DB-GPT/examples/awel/simple_chat_dag_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_chat_history_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=dbgpt_awel_simple_chat_history) from mod <module 'unusual_prefix_df6ef9efa85c05bbcf0a6b504b28120f41c43c44_simple_chat_history_example' from '/home/suc/DB-GPT/examples/awel/simple_chat_history_example.py'> and model file /home/suc/DB-GPT/examples/awel/simple_chat_history_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_dag_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=simple_dag_example) from mod <module 'unusual_prefix_1039439a300b70e0504e8a524cbf87e52d14564f_simple_dag_example' from '/home/suc/DB-GPT/examples/awel/simple_dag_example.py'> and model file /home/suc/DB-GPT/examples/awel/simple_dag_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_llm_client_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_generate) from mod <module 'unusual_prefix_d16d9c43045ead23635427d2c00d589c70d6ec60_simple_llm_client_example' from '/home/suc/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /home/suc/DB-GPT/examples/awel/simple_llm_client_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_count_token) from mod <module 'unusual_prefix_d16d9c43045ead23635427d2c00d589c70d6ec60_simple_llm_client_example' from '/home/suc/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /home/suc/DB-GPT/examples/awel/simple_llm_client_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] ERROR Failed to import: /home/suc/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py, error message: Traceback (most recent call last):
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/core/awel/dag/loader.py", line 91, in parse
loader.exec_module(new_module)
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/suc/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py", line 235, in
llm = OpenAILLMClient()
^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 178, in init
_ = self.client.default_headers
^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 229, in client
self._api_type, self._client = _build_openai_client(
^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 103, in _build_openai_client
openai_params, api_type, api_version, api_azure_deployment = _initialize_openai_v1(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 90, in _initialize_openai_v1
raise ValueError("api_key is required, please set OPENAI_API_KEY environment")
ValueError: api_key is required, please set OPENAI_API_KEY environment

2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] ERROR Failed to import: /home/suc/DB-GPT/examples/awel/simple_rag_rewrite_example.py, error message: Traceback (most recent call last):
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/core/awel/dag/loader.py", line 91, in parse
loader.exec_module(new_module)
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/suc/DB-GPT/examples/awel/simple_rag_rewrite_example.py", line 64, in
rewrite_task = QueryRewriteOperator(llm_client=OpenAILLMClient(), nums=2)
^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 178, in init
_ = self.client.default_headers
^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 229, in client
self._api_type, self._client = _build_openai_client(
^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 103, in _build_openai_client
openai_params, api_type, api_version, api_azure_deployment = _initialize_openai_v1(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 90, in _initialize_openai_v1
raise ValueError("api_key is required, please set OPENAI_API_KEY environment")
ValueError: api_key is required, please set OPENAI_API_KEY environment

2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] INFO Importing /home/suc/DB-GPT/examples/awel/simple_rag_summary_example.py
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.dag.loader[6384] ERROR Failed to import: /home/suc/DB-GPT/examples/awel/simple_rag_summary_example.py, error message: Traceback (most recent call last):
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/core/awel/dag/loader.py", line 91, in parse
loader.exec_module(new_module)
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/suc/DB-GPT/examples/awel/simple_rag_summary_example.py", line 65, in
llm_client=OpenAILLMClient(), language="en"
^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 178, in init
_ = self.client.default_headers
^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/proxy/llms/chatgpt.py", line 229, in client
self._api_type, self._client = _build_openai_client(
^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 103, in _build_openai_client
openai_params, api_type, api_version, api_azure_deployment = _initialize_openai_v1(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/utils/chatgpt_utils.py", line 90, in _initialize_openai_v1
raise ValueError("api_key is required, please set OPENAI_API_KEY environment")
ValueError: api_key is required, please set OPENAI_API_KEY environment

2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=1cb620cb-2cd5-41eb-835e-985751a0f5de)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fd48f3b27a0>(AWEL_trigger_route__examples_data_analyst_copilot), endpoint: /examples/data_analyst/copilot, methods: ['POST']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/data_analyst/copilot
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=b86db1cd-3c9b-4141-b08d-98dadfa25704)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fd48f3b2c00>(AWEL_trigger_route__examples_simple_chat), endpoint: /examples/simple_chat, methods: ['POST']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_chat
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=bc7b3757-8a51-4bcd-aba0-08c92a46c867)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fd48f3b34c0>(AWEL_trigger_route__examples_simple_history_multi_round_chat_completions), endpoint: /examples/simple_history/multi_round/chat/completions, methods: ['POST']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_history/multi_round/chat/completions
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=95a85265-5160-4520-a06b-633d4dc05b66)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function_get at 0x7fd48f3b0fe0>(AWEL_trigger_route__examples_hello), endpoint: /examples/hello, methods: ['GET']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/hello
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=dc0b6eaf-3217-4d18-86f6-e4f68aae505b)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fd48f410360>(AWEL_trigger_route__examples_simple_client_chat_completions), endpoint: /examples/simple_client/chat/completions, methods: ['POST']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/chat/completions
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Register trigger HttpTrigger(node_id=bfcbfa49-0f35-45dc-b14b-08ead2cbc806)
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fd48f411080>(AWEL_trigger_route__examples_simple_client_count_token), endpoint: /examples/simple_client/count_token, methods: ['POST']
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.http_trigger[6384] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/count_token
2025-03-10 21:27:39 localhost.localdomain dbgpt.core.awel.trigger.trigger_manager[6384] INFO Include router <fastapi.routing.APIRouter object at 0x7fd49b115e50> to prefix path /api/v1/awel/trigger
Libro Server start!
start libro exception！[Errno 2] No such file or directory: 'libro'
2025-03-10 21:27:39 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Begin start all worker, apply_req: None
2025-03-10 21:27:39 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.._start_worker at 0x7fd48f28cea0>
2025-03-10 21:27:39 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Apply to all workers
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:5670 (Press CTRL+C to quit)
2025-03-10 21:27:39 localhost.localdomain dbgpt.model.cluster.worker.embedding_worker[6384] INFO Load embeddings model: BAAI/bge-large-zh-v1.5
2025-03-10 21:27:41 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
INFO: 127.0.0.1:36232 - "POST /api/controller/models HTTP/1.1" 200 OK
2025-03-10 21:27:41 localhost.localdomain dbgpt.model.cluster.worker.embedding_worker[6384] INFO Load rerank embeddings model: BAAI/bge-reranker-v2-m3
2025-03-10 21:27:43 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
2025-03-10 21:27:43 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
2025-03-10 21:27:43 localhost.localdomain dbgpt.util.code.server[6384] INFO Code server is ready
INFO: 127.0.0.1:56006 - "POST /api/controller/models HTTP/1.1" 200 OK
2025-03-10 21:27:43 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] ERROR Error starting worker manager: model Qwen/Qwen2.5-Coder-32B-Instruct@proxy/siliconflow(192.168.3.25:5670) start failed, Traceback (most recent call last):
File "/home/suc/DB-GPT/.venv/lib/python3.11/site-packages/torch/init.py", line 300, in _load_global_deps
ctypes.CDLL(global_deps_lib_path, mode=ctypes.RTLD_GLOBAL)
File "/home/suc/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11/ctypes/init.py", line 376, in init
self._handle = _dlopen(self._name, mode)
^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: libcudart.so.12: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/cluster/worker/manager.py", line 631, in _start_worker
await self.run_blocking_func(
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/cluster/worker/manager.py", line 146, in run_blocking_func
return await loop.run_in_executor(self.executor, func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/cluster/worker/default_worker.py", line 102, in start
_try_import_torch()
File "/home/suc/DB-GPT/packages/dbgpt-core/src/dbgpt/model/cluster/worker/default_worker.py", line 626, in _try_import_torch
import torch
File "/home/suc/DB-GPT/.venv/lib/python3.11/site-packages/torch/init.py", line 366, in
_load_global_deps()
File "/home/suc/DB-GPT/.venv/lib/python3.11/site-packages/torch/init.py", line 324, in _load_global_deps
_preload_cuda_deps(lib_folder, lib_name)
File "/home/suc/DB-GPT/.venv/lib/python3.11/site-packages/torch/init.py", line 284, in _preload_cuda_deps
raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.[0-9] not found in the system path ['.', '.', '.', '.', '/home/suc/DB-GPT/.venv/bin', '/home/suc/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python311.zip', '/home/suc/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11', '/home/suc/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11/lib-dynload', '/home/suc/DB-GPT/.venv/lib/python3.11/site-packages', '/home/suc/DB-GPT/packages/dbgpt-core/src', '/home/suc/DB-GPT/packages/dbgpt-app/src', '/home/suc/DB-GPT/packages/dbgpt-client/src', '/home/suc/DB-GPT/packages/dbgpt-ext/src', '/home/suc/DB-GPT/packages/dbgpt-serve/src', '/home/suc/DB-GPT/packages/dbgpt-app', '/home/suc/DB-GPT/packages/dbgpt-app']
;model BAAI/bge-large-zh-v1.5@proxy/openai(192.168.3.25:5670) start successfully
;model BAAI/bge-reranker-v2-m3@proxy/siliconflow(192.168.3.25:5670) start successfully
INFO: Shutting down
INFO: Waiting for application shutdown.
2025-03-10 21:27:43 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Stop all workers
2025-03-10 21:27:43 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.._stop_worker at 0x7fd48f28cfe0>
2025-03-10 21:27:43 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] INFO Apply to all workers
2025-03-10 21:27:43 localhost.localdomain dbgpt.model.cluster.worker.default_worker[6384] WARNING Model has been stopped!!
2025-03-10 21:27:43 localhost.localdomain dbgpt.util.api_utils[6384] WARNING Health check failed for http://127.0.0.1:5670, error: HTTPConnectionPool(host='127.0.0.1', port=5670): Max retries exceeded with url: /api/health (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd48f2e5590>: Failed to establish a new connection: [Errno 111] Connection refused'))
2025-03-10 21:27:45 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
2025-03-10 21:27:45 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] WARNING Send heartbeat func error: All connection attempts failed
2025-03-10 21:27:45 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
2025-03-10 21:27:45 localhost.localdomain dbgpt.util.api_utils[6384] WARNING No healthy urls found, selecting randomly
2025-03-10 21:27:45 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed
2025-03-10 21:27:45 localhost.localdomain dbgpt.model.cluster.worker.manager[6384] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed
INFO: Application shutdown complete.
INFO: Finished server process [6384]
in order to avoid chroma db atexit problem