- Fetch repository info.
- Fetch user info.
- Install Ollama and Llama3.
- Create a python virtual env and install the requirements.
- run
python main.py
to create the git model. - run
python tests.py
to run all the tests.
In this demo, we can see that running the code generator LLM for the same tests ran successfully before but fails for one test case when ran again. This is because of code generation variance and hallucinations.
In this case, the LLM generated and ran the get all users route instead of get user by username route.