feat: add expression helpers for parsing LoadSubsetOptions in queryFn #786

KyleAMathews · 2025-11-10T17:17:00Z

Summary

Adds reusable expression parsing utilities to help developers translate TanStack DB predicates (where, orderBy, limit) into their API's format when implementing query collections with predicate push-down.

Changes

New Expression Helpers (`packages/query-db-collection/src/expression-helpers.ts`)

parseWhereExpression: Parse where clauses with custom handlers for each operator
parseOrderByExpression: Parse order by into simple array format
extractSimpleComparisons: Extract simple AND-ed filters for basic use cases
parseLoadSubsetOptions: Convenience function to parse all options at once
walkExpression, extractFieldPath, extractValue: Lower-level helpers for custom parsing

Documentation

Added comprehensive section to docs/collections/query-collection.md:

How LoadSubsetOptions are passed via ctx.meta.loadSubsetOptions
Expression helper usage with REST API examples
GraphQL example
API reference for all helper functions
Query key builder pattern
Tips and best practices

Tests

35 test cases with 93.65% coverage:

All helper functions tested
Various operator combinations (eq, lt, gt, and, or, etc.)
Edge cases (null/undefined, nested expressions)
Integration test with complex real-world query

Why This Matters

Without these helpers, developers had to manually parse expression AST trees:

// Before - complex and error-prone
if (where && where.type === 'func' && where.name === 'eq') {
  const field = where.args[0].path[0]
  const value = where.args[1].value
  // ... more traversal logic
}

With helpers, it's straightforward:

// After - simple and maintainable
const filters = parseWhereExpression(where, {
  handlers: {
    eq: (field, value) => ({ [field.join('.')]: value }),
    lt: (field, value) => ({ [`${field.join('.')}_lt`]: value }),
    and: (...conditions) => Object.assign({}, ...conditions)
  }
})

Example Usage

queryFn: async (ctx) => {
  const { where, orderBy, limit } = ctx.meta.loadSubsetOptions

  // Parse into your API format
  const parsed = parseLoadSubsetOptions({ where, orderBy, limit })

  // Build API request
  const params = new URLSearchParams()
  parsed.filters.forEach(({ field, operator, value }) => {
    if (operator === 'eq') {
      params.set(field.join('.'), String(value))
    }
  })

  return fetch(`/api/products?${params}`).then(r => r.json())
}

🤖 Generated with Claude Code

Add reusable expression parsing utilities to help developers translate TanStack DB predicates (where, orderBy, limit) into their API's format. - Add expression-helpers.ts with generic parsing utilities - parseWhereExpression: Parse where clauses with custom handlers - parseOrderByExpression: Parse order by into simple array - extractSimpleComparisons: Extract simple AND-ed filters - parseLoadSubsetOptions: Convenience function for all options - walkExpression, extractFieldPath, extractValue: Lower-level helpers - Export helpers from query-db-collection package - Add comprehensive documentation to query-collection.md - How LoadSubsetOptions are passed via ctx.meta - Expression helper usage with REST and GraphQL examples - API reference for all helper functions - Tips and best practices This makes it much easier to implement query collections with predicate push-down without having to manually parse expression AST trees. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

- 35 test cases covering all helper functions - Tests for parseWhereExpression with various operators - Tests for parseOrderByExpression with sorting options - Tests for extractSimpleComparisons - Tests for parseLoadSubsetOptions - Tests for low-level helpers (extractFieldPath, extractValue, walkExpression) - Integration test for complex real-world query scenarios All tests passing with 93.65% coverage of expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

changeset-bot · 2025-11-10T17:17:04Z

🦋 Changeset detected

Latest commit: d0377c6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages

Name	Type
@tanstack/db	Patch
@tanstack/query-db-collection	Patch
@tanstack/angular-db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/powersync-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/rxdb-db-collection	Patch
@tanstack/solid-db	Patch
@tanstack/svelte-db	Patch
@tanstack/trailbase-db-collection	Patch
@tanstack/vue-db	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Remove Hasura-specific emphasis in GraphQL example section. The underscore-prefixed operators are common GraphQL conventions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Change 'the GraphQL format' to 'a GraphQL format' to avoid implying this is the only way to structure GraphQL queries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

Remove implementation-specific reference to Electric DB Collection from the tips section. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

- Add default empty object parameter to createQueryFromOpts - Remove ?? {} pattern from all documentation - Update changeset example - Update PR description This makes the API cleaner - users can now access ctx.meta.loadSubsetOptions directly without null coalescing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

- Import BasicExpression and OrderBy from IR namespace - Add type annotations to all lambda parameters - Remove unnecessary type checks that were always true - Fix variable shadowing in walkExpression visitor parameter - Fixes build type errors and eslint warnings in expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

pkg-pr-new · 2025-11-10T17:40:17Z

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@786

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@786

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@786

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@786

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@786

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@786

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@786

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@786

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@786

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@786

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@786

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@786

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@786

commit: d0377c6

github-actions · 2025-11-10T17:40:53Z

Size Change: +1.4 kB (+1.67%)

Total Size: 85.5 kB

Filename	Size	Change
`./packages/db/dist/esm/index.js`	2.63 kB	+139 B (+5.58%)	🔍
`./packages/db/dist/esm/query/builder/functions.js`	733 B	+127 B (+20.96%)	🚨
`./packages/db/dist/esm/query/expression-helpers.js`	1.14 kB	+1.14 kB (new file)	🆕

ℹ️ View Unchanged

Filename	Size
`./packages/db/dist/esm/collection/change-events.js`	1.38 kB
`./packages/db/dist/esm/collection/changes.js`	977 B
`./packages/db/dist/esm/collection/events.js`	388 B
`./packages/db/dist/esm/collection/index.js`	3.24 kB
`./packages/db/dist/esm/collection/indexes.js`	1.1 kB
`./packages/db/dist/esm/collection/lifecycle.js`	1.67 kB
`./packages/db/dist/esm/collection/mutations.js`	2.26 kB
`./packages/db/dist/esm/collection/state.js`	3.43 kB
`./packages/db/dist/esm/collection/subscription.js`	2.42 kB
`./packages/db/dist/esm/collection/sync.js`	2.12 kB
`./packages/db/dist/esm/deferred.js`	207 B
`./packages/db/dist/esm/errors.js`	4.11 kB
`./packages/db/dist/esm/event-emitter.js`	748 B
`./packages/db/dist/esm/indexes/auto-index.js`	742 B
`./packages/db/dist/esm/indexes/base-index.js`	766 B
`./packages/db/dist/esm/indexes/btree-index.js`	1.87 kB
`./packages/db/dist/esm/indexes/lazy-index.js`	1.1 kB
`./packages/db/dist/esm/indexes/reverse-index.js`	513 B
`./packages/db/dist/esm/local-only.js`	837 B
`./packages/db/dist/esm/local-storage.js`	2.04 kB
`./packages/db/dist/esm/optimistic-action.js`	359 B
`./packages/db/dist/esm/paced-mutations.js`	496 B
`./packages/db/dist/esm/proxy.js`	3.22 kB
`./packages/db/dist/esm/query/builder/index.js`	3.84 kB
`./packages/db/dist/esm/query/builder/ref-proxy.js`	917 B
`./packages/db/dist/esm/query/compiler/evaluators.js`	1.34 kB
`./packages/db/dist/esm/query/compiler/expressions.js`	674 B
`./packages/db/dist/esm/query/compiler/group-by.js`	1.8 kB
`./packages/db/dist/esm/query/compiler/index.js`	1.96 kB
`./packages/db/dist/esm/query/compiler/joins.js`	2 kB
`./packages/db/dist/esm/query/compiler/order-by.js`	1.25 kB
`./packages/db/dist/esm/query/compiler/select.js`	1.07 kB
`./packages/db/dist/esm/query/ir.js`	673 B
`./packages/db/dist/esm/query/live-query-collection.js`	360 B
`./packages/db/dist/esm/query/live/collection-config-builder.js`	5.26 kB
`./packages/db/dist/esm/query/live/collection-registry.js`	264 B
`./packages/db/dist/esm/query/live/collection-subscriber.js`	1.77 kB
`./packages/db/dist/esm/query/live/internal.js`	130 B
`./packages/db/dist/esm/query/optimizer.js`	2.6 kB
`./packages/db/dist/esm/query/predicate-utils.js`	2.88 kB
`./packages/db/dist/esm/query/subset-dedupe.js`	921 B
`./packages/db/dist/esm/scheduler.js`	1.21 kB
`./packages/db/dist/esm/SortedMap.js`	1.18 kB
`./packages/db/dist/esm/strategies/debounceStrategy.js`	237 B
`./packages/db/dist/esm/strategies/queueStrategy.js`	418 B
`./packages/db/dist/esm/strategies/throttleStrategy.js`	236 B
`./packages/db/dist/esm/transactions.js`	2.9 kB
`./packages/db/dist/esm/utils.js`	881 B
`./packages/db/dist/esm/utils/browser-polyfills.js`	304 B
`./packages/db/dist/esm/utils/btree.js`	5.61 kB
`./packages/db/dist/esm/utils/comparison.js`	660 B
`./packages/db/dist/esm/utils/index-optimization.js`	1.51 kB
`./packages/db/dist/esm/utils/type-guards.js`	157 B

_{compressed-size-action::db-package-size}

github-actions · 2025-11-10T17:41:29Z

Size Change: 0 B

Total Size: 3.34 kB

ℹ️ View Unchanged

Filename	Size
`./packages/react-db/dist/esm/index.js`	225 B
`./packages/react-db/dist/esm/useLiveInfiniteQuery.js`	1.17 kB
`./packages/react-db/dist/esm/useLiveQuery.js`	1.11 kB
`./packages/react-db/dist/esm/useLiveSuspenseQuery.js`	431 B
`./packages/react-db/dist/esm/usePacedMutations.js`	401 B

_{compressed-size-action::react-db-package-size}

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

samwillis · 2025-11-10T18:01:18Z

packages/db/src/query/expression-helpers.ts

+  handlers: {
+    [operator: string]: (...args: Array<any>) => T
+  }


can we improve the type here such that it knows the operators that the IR supports?

samwillis · 2025-11-10T18:04:15Z

packages/query-db-collection/src/expression-helpers.ts

+/**
+ * Represents a simple field path extracted from an expression
+ */
+export type FieldPath = Array<string>


We may want to support accessing elements in a array:

Suggested change

export type FieldPath = Array<string>

export type FieldPath = Array<string | number>

samwillis · 2025-11-10T18:05:51Z

packages/db/src/query/expression-helpers.ts

+ * // ]
+ * ```
+ */
+export function extractSimpleComparisons(


We may want this to throw if if finds something it can't extract such as an or

samwillis · 2025-11-10T18:19:15Z

packages/query-db-collection/src/expression-helpers.ts

+    return {
+      field,
+      direction: clause.compareOptions.direction,
+      nulls: clause.compareOptions.nulls,
+    }


we need to return the local / string comparison options too

samwillis · 2025-11-10T18:24:07Z

Mostly there except for the above comments. I will rebase #773 on this and use it.

samwillis · 2025-11-10T19:17:51Z

@KyleAMathews there are a few tests failing too, looks like just types

- Add OperatorName type and operators constant to db package - Improve ParseWhereOptions handlers type to know IR operators - Support array indices in FieldPath (string | number) - Add locale/string collation options to ParsedOrderBy - Make extractSimpleComparisons throw on unsupported operations (or, not, etc.) - Update tests to expect throws instead of silent skipping

CompareOptions uses StringCollationConfig which has stringSort, locale, and localeOptions fields, not sensitivity and locale at the top level.

Use 'in' operator to check for optional fields (locale, localeOptions) that only exist when stringSort is 'locale'

samwillis

Looks great. I'll merge it into #763

KyleAMathews · 2025-11-11T14:46:39Z

Cool! BTW, I've been thinking we'll probably want to move something like these utilities into a shared package as other collections will surely want to do something similar. The babel of the IR haha.

…#786) * feat: add expression helpers for parsing LoadSubsetOptions in queryFn Add reusable expression parsing utilities to help developers translate TanStack DB predicates (where, orderBy, limit) into their API's format. - Add expression-helpers.ts with generic parsing utilities - parseWhereExpression: Parse where clauses with custom handlers - parseOrderByExpression: Parse order by into simple array - extractSimpleComparisons: Extract simple AND-ed filters - parseLoadSubsetOptions: Convenience function for all options - walkExpression, extractFieldPath, extractValue: Lower-level helpers - Export helpers from query-db-collection package - Add comprehensive documentation to query-collection.md - How LoadSubsetOptions are passed via ctx.meta - Expression helper usage with REST and GraphQL examples - API reference for all helper functions - Tips and best practices This makes it much easier to implement query collections with predicate push-down without having to manually parse expression AST trees. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * test: add comprehensive tests for expression helpers - 35 test cases covering all helper functions - Tests for parseWhereExpression with various operators - Tests for parseOrderByExpression with sorting options - Tests for extractSimpleComparisons - Tests for parseLoadSubsetOptions - Tests for low-level helpers (extractFieldPath, extractValue, walkExpression) - Integration test for complex real-world query scenarios All tests passing with 93.65% coverage of expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * chore: add changeset for expression helpers 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: make GraphQL example more generic Remove Hasura-specific emphasis in GraphQL example section. The underscore-prefixed operators are common GraphQL conventions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: clarify GraphQL format is not prescriptive Change 'the GraphQL format' to 'a GraphQL format' to avoid implying this is the only way to structure GraphQL queries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * docs: remove Electric DB reference from tips Remove implementation-specific reference to Electric DB Collection from the tips section. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * refactor: make loadSubsetOptions always an object - Add default empty object parameter to createQueryFromOpts - Remove ?? {} pattern from all documentation - Update changeset example - Update PR description This makes the API cleaner - users can now access ctx.meta.loadSubsetOptions directly without null coalescing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: add explicit type annotations to avoid implicit any - Import BasicExpression and OrderBy from IR namespace - Add type annotations to all lambda parameters - Remove unnecessary type checks that were always true - Fix variable shadowing in walkExpression visitor parameter - Fixes build type errors and eslint warnings in expression-helpers.ts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * style: fix prettier formatting in changeset 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]> * fix: correct TypeScript types in expression-helpers tests * feat: address Sam's review feedback - Add OperatorName type and operators constant to db package - Improve ParseWhereOptions handlers type to know IR operators - Support array indices in FieldPath (string | number) - Add locale/string collation options to ParsedOrderBy - Make extractSimpleComparisons throw on unsupported operations (or, not, etc.) - Update tests to expect throws instead of silent skipping * fix: remove invalid sensitivity/locale fields from test mocks CompareOptions uses StringCollationConfig which has stringSort, locale, and localeOptions fields, not sensitivity and locale at the top level. * fix: handle discriminated union for CompareOptions properly Use 'in' operator to check for optional fields (locale, localeOptions) that only exist when stringSort is 'locale' * move helpers to main db package --------- Co-authored-by: Claude <[email protected]> Co-authored-by: Sam Willis <[email protected]>

KyleAMathews and others added 2 commits November 10, 2025 09:08

KyleAMathews and others added 6 commits November 10, 2025 09:19

chore: add changeset for expression helpers

d2ca26e

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

docs: make GraphQL example more generic

6bffd23

Remove Hasura-specific emphasis in GraphQL example section. The underscore-prefixed operators are common GraphQL conventions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

docs: clarify GraphQL format is not prescriptive

c2f47e2

Change 'the GraphQL format' to 'a GraphQL format' to avoid implying this is the only way to structure GraphQL queries. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

docs: remove Electric DB reference from tips

9677f86

Remove implementation-specific reference to Electric DB Collection from the tips section. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

style: fix prettier formatting in changeset

a273bb1

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

KyleAMathews force-pushed the queryfn-docs branch from 302495f to a273bb1 Compare November 10, 2025 17:45

samwillis requested changes Nov 10, 2025

View reviewed changes

samwillis mentioned this pull request Nov 10, 2025

Query Driven Sync with On-Demand Collection Loading #763

Merged

7 tasks

KyleAMathews added 3 commits November 10, 2025 13:30

fix: correct TypeScript types in expression-helpers tests

1146f49

fix: remove invalid sensitivity/locale fields from test mocks

8648e52

CompareOptions uses StringCollationConfig which has stringSort, locale, and localeOptions fields, not sensitivity and locale at the top level.

KyleAMathews requested a review from samwillis November 10, 2025 21:38

fix: handle discriminated union for CompareOptions properly

a32b363

Use 'in' operator to check for optional fields (locale, localeOptions) that only exist when stringSort is 'locale'

samwillis approved these changes Nov 11, 2025

View reviewed changes

samwillis mentioned this pull request Nov 11, 2025

E2E Test Suite for Query-Driven Sync #773

Merged

7 tasks

move helpers to main db package

d0377c6

samwillis merged commit f7b3988 into query-driven-sync Nov 11, 2025
6 of 7 checks passed

samwillis deleted the queryfn-docs branch November 11, 2025 23:58

	export type FieldPath = Array<string>
	export type FieldPath = Array<string \| number>

feat: add expression helpers for parsing LoadSubsetOptions in queryFn #786

feat: add expression helpers for parsing LoadSubsetOptions in queryFn #786

Uh oh!

Conversation

KyleAMathews commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New Expression Helpers (packages/query-db-collection/src/expression-helpers.ts)

Documentation

Tests

Why This Matters

Example Usage

Uh oh!

changeset-bot bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samwillis Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

samwillis Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

samwillis Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

samwillis Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

samwillis commented Nov 10, 2025

Uh oh!

samwillis commented Nov 10, 2025

Uh oh!

samwillis left a comment

Choose a reason for hiding this comment

Uh oh!

KyleAMathews commented Nov 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

KyleAMathews commented Nov 10, 2025 •

edited

Loading

New Expression Helpers (`packages/query-db-collection/src/expression-helpers.ts`)

changeset-bot bot commented Nov 10, 2025 •

edited

Loading

pkg-pr-new bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading

github-actions bot commented Nov 10, 2025 •

edited

Loading