v3.2.0

3.2.0 (2024-05-10)

Features

• better api support for node-llama-cpp@beta (0ecce70)

Development API + Node-llama-cpp@beta integration

You can use the model with node-llama-cpp@beta

CatAI enables you to easily manage the models and chat with them.

import {downloadModel, getModelPath} from 'catai';

// download the model, skip if you already have the model
await downloadModel(
    "https://huggingface.co/QuantFactory/Meta-Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct.Q2_K.gguf?download=true",
    "llama3"
);

// get the model path with catai
const modelPath = getModelPath("llama3");

const llama = await getLlama();
const model = await llama.loadModel({
    modelPath
});

const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});

const a1 = await session.prompt("Hi there, how are you?");
console.log("AI: " + a1);