use models specific to LightRAG/NanoGraphRAG processing

[vap0rtranz]

How could models trained for GraphRAG/Light/Nano processing be integrated with Kotaemon?

I've found that the generic models, like Llama3 and Qwen2, are slow even on GPU.

Models have been trained for GraphRAG processing. For example, Triplex. Triplex authors claim it is far more efficient.

How can the app's settings be changed to add a GraphRAG specific model to run locally?

I'm thinking a workflow like the setup for Embeddings to use embed specific models via Ollama.

[vap0rtranz]

Just to clarify, I looks like the model used for the graph processing is hardcoded. I've tried a few reconfgurations in the UI thinking it would change the model, like changing the default LLM. LightRAG always uses Llama3.2 1B.

Oddly enough, my local Ollama doesn't even have the Llama-3.2-1B-Instruct pulled to run. ....

Logs:

INFO:lightrag:Logger initialized for working directory: /app/ktem_app_data/user_data/files/lightrag/134135b5-3eba-4164-9fa2-d49d71382336/input
DEBUG:lightrag:LightRAG init with param:
  working_dir = /app/ktem_app_data/user_data/files/lightrag/134135b5-3eba-4164-9fa2-d49d71382336/input,
  chunk_token_size = 1200,
  chunk_overlap_token_size = 100,
  tiktoken_model_name = gpt-4o-mini,
  entity_extract_max_gleaning = 1,
  entity_summary_to_max_tokens = 500,
  node_embedding_algorithm = node2vec,
  node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3},
  embedding_func = {'embedding_dim': 768, 'max_token_size': 8192, 'func': <function get_embedding_func.<locals>.embedding_func at 0x7f1a40ea2dd0>},
  embedding_batch_num = 32,
  embedding_func_max_async = 16,
  llm_model_func = <function get_llm_func.<locals>.llm_func at 0x7f1a40ea2e60>,
  llm_model_name = meta-llama/Llama-3.2-1B-Instruct,
  llm_model_max_token_size = 32768,
  llm_model_max_async = 16,
  key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>,
  vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>,
  vector_db_storage_cls_kwargs = {},
  graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>,
  enable_llm_cache = True,
  addon_params = {},
  convert_response_to_json_func = <function convert_response_to_json at 0x7f19f7d63d90>