Bug: Error: Buffer manager exception: Unable to allocate memory! The buffer pool is full and no memory could be freed!

[bgoosmanviz]

Kùzu version

v0.7.1

What operating system are you using?

MacOS

What happened?

Running into "Error: Buffer manager exception: Unable to allocate memory! The buffer pool is full and no memory could be freed!" with kuzu-wasm. Wondering if there is a workaround. Here's my function.

export async function kuzuQuery<T>(path: string, query: string, params = {}) {
  const result = await kuzuWasmDbManager.query<T>(path, query, params);
  const rows = await result.getAllObjects();
  await result.close();
  return { content: rows, status: 0 };
}

kuzuWasmDbManager keeps the Database and Connection in-memory for a long time. The query I can't run more than once.

MATCH path=(a)-[r*1..5]-(b)
WHERE a.id IN ["A", "B", "C"]
AND b.id IN ["A", "B", "C"]
AND a.id <> b.id
AND is_acyclic(path)=true
RETURN path;

I thought closing the QueryResult would be enough, but no luck.

Here's my query function:

  async query<T>(path: string, query: string, params: KuzuQueryParams = {}): Promise<KuzuQueryResult<T>> {
    await this.pruneDatabases();
    const conn = await this.getConnection(path);
    const state = this.state.get(path);
    if (state) {
      state.lastAccessed = Date.now();
      this.state.set(path, state);
    }
    const preparedStatement = await conn.prepare(query);
    const result = await conn.execute<T>(preparedStatement, params);
    return result;
  }

Are there known steps to reproduce?

No response

[andyfengHKU]

Hi @bgoosmanviz,

I wonder if it's because the recursive pattern is generating a lot of intermediate result causing buffer manager to go out of memory. Do u mind try the following

MATCH path=(a)-[r* TRAIL 1..5]-(b)
WHERE a.id IN ["A", "B", "C"]
AND b.id IN ["A", "B", "C"]
AND a.id <> b.id
RETURN path;

By specifying TRAIL in recursive pattern we should reduce the intermediate result by a lot if there are many cycles. If not, we need to look more closely with a dataset.

[bgoosmanviz]

Thanks @andyfengHKU I'll give it a try

[bgoosmanviz]

So that query does indeed appear to work. I'll keep an eye out for any memory allocation errors.