doculoom-server

An LLM based contextual question answering chatbot

Local setup

Prerequisites to use local models (LLAMA/Mistral)

1. Install Ollama
2. Start the server
3. Run ollama pull llama3.2:1b

Prerequisites to use OpenAI models

1. Set openai env var

export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

2. Set default model to LLAMA3_2_1B

export MODEL=LLAMA3_2_1B

Install dependencies

pip install poetry && poetry install --no-dev

Start the server

poetry run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Docker setup

To run the server, follow these steps:

1. Build the Docker Image

   docker build -t doculoom .

2. Start the server

   docker run -it --rm -p 8000:8000 doculoom