Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gte is 1000+ times slower than plain filter #895

Open
mtsmfm opened this issue Feb 25, 2025 · 0 comments
Open

gte is 1000+ times slower than plain filter #895

mtsmfm opened this issue Feb 25, 2025 · 0 comments

Comments

@mtsmfm
Copy link

mtsmfm commented Feb 25, 2025

Describe the bug

search(db, { where: { n: { gte: 1 } } }) takes 500ms but numbers.filter((n) => n >= 1).length can be finished within 0.3ms

To Reproduce

Run the following snippet.

import { create, insertMultiple, search } from "@orama/orama";

const N = 10000;
const main = async () => {
  const numbers = Array.from({ length: N }, (_, i) => i);

  const db = create({
    schema: {
      n: "number",
    },
  });
  insertMultiple(
    db,
    numbers.map((n) => ({ n })),
    numbers.length
  );

  console.time("search");
  const result1 = await search(db, { where: { n: { gte: 1 } } });
  console.timeEnd("search");

  console.time("filter");
  const result2 = numbers.filter((n) => n >= 1).length;
  console.timeEnd("filter");

  if (result1.count !== N - 1) {
    throw new Error("invalid count");
  }
  if (result2 !== N - 1) {
    throw new Error("invalid count");
  }
};

main();

/*
$ NODE_NO_WARNINGS=1 node --experimental-strip-types test.ts
search: 588.004ms
filter: 0.241ms
*/

Expected behavior

search should be finished at the latest the same-ish speed. Considering Orama can create sorted index, it's preferable to be finished faster than entire search though.

Environment Info

System:
    OS: Linux 5.15 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
    CPU: (12) x64 AMD Ryzen 5 3600 6-Core Processor
    Memory: 8.59 GB / 15.57 GB
    Container: Yes
  Binaries:
    Node: 22.13.0 - ~/.local/share/mise/installs/node/22/bin/node
    npm: 11.1.0 - ~/.local/share/mise/installs/node/22/bin/npm

Affected areas

Search

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant