Use llama.dll for my game's NPCs

[ninjadynamics]

Hello, AI noob here!

I'm writing a small game in C (with RayLib) where I'd like my NPCs to have "smart" conversations with the player, using llamacpp as a backend.

So far, I've compiled llamacpp into a shared library using make llama.dll and asked ChatGPT (o1-preview) to generate a C program that emulates llama-cli.exe with limited success. The code works fine, but after some back and forth interactions, the model starts to speak nonsense/partial tokens and doesn't remember what we're talking about...

I'm using w64devkit on Windows 10 and my compile command is:
~/llama.cpp $ gcc chat.c -o chat.exe -I./include -I./ggml/include -L. -lllama -lstdc++ -pthread -lm

Note: chat.c exists on the same directory as llama-cli.exe

The model I'm using is llama-3.2-1b-instruct-q8_0.gguf which works pretty well with llama-cli.exe.

Here's my chat.c code:
https://gist.github.com/ninjadynamics/6d1eb01401dd6822293940676e84decd

Can anyone help?

Many thanks in advance!

[Zapotecatl]

In case it helps.

I made my NPC in UE4 with llamacpp (Game).

I used the example of main

main example

and I put it in a UE4 thread with the structure of sending thread to sleep and waking thread up when the NPC has to talk.

https://unrealcommunity.wiki/6100e8169c9d1a89e0c3458f

[bertsons]

@Zapotecatl That's pretty cool, I'll be sure to give that game a try. I am very interested in integrating llama.cpp with Unreal Engine, and have some projects in this area. Do you know of other (open source) examples integrating llama.cpp with UE?

[Zapotecatl]

Great!

Sorry, I don't know of any games that use llama.cpp, UE, and are open source.

The only game I've seen that also claims to use llama.cpp is this one (not unreal and not open source):
hammerai-desktop