Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Set operations (union / intersect / except) support #1218

Merged
merged 36 commits into from
Nov 2, 2023

Conversation

Angelelz
Copy link
Collaborator

This PR will close #207

Added a new file set-operators.ts with a SetOperatorQueryBuilder class and a SetOperator class

The MySqlSelectBuilder now extends the SetOperatorQueryBuilder to inherit the set operator methods

Added the export for the set operation functions
This implementation will likely change with the same pattern used in the Mysql implementation
… the user attempts to select different types on both sides of the set operator
…or functions. The types might need some more work
…perator functions

 - Added a new type 'SetOperatorRestSelect' to properly handle the rest parameter

 - Added a new generic parameter TRest to properly handle the rest parameter

 - Added additional type tests for the rest parameters

 - deleted debigging strings on ValidateShape type
 - fixed wring import in pg that prevented test from running

 - added tests
 - Added abstract class PgSetOperatorBuilder that extends the TypedQueryBuilder

 - Added type helpers for correct type inference including when passing multiple parameters to the function form

 - PgSelect now extends from PgSetOperatorBuilder instead of TypedQueryBuilder to inherit all its methods
 - added SQLiteSetOperatorBuilder and SQLiteSetOperator classes

 - SQLiteSelectBuilder now extends from SQLiteSetOperatorBuilder to inherit its methods

 - Exported funciton versions of union, unionAll, intersect and except

 - Had to add the last generic parameter as any to the in a type for libsql driver
@Angelelz Angelelz marked this pull request as ready for review September 21, 2023 06:19
@Angelelz
Copy link
Collaborator Author

Angelelz commented Sep 21, 2023

In its most basic form, upon merging this PR, a drizzle user will be able to create queries with union / intersect / except {all} statements with the following syntax:

await db.select().from(table)
  .union(
    db.select().from(table)
  );

Beyond that, given the following queries:

const query1 = db
  .select({ id: users.id, name: users.name })
  .from(users)
  .where(eq(users.id, 1))
  .leftJoin(posts, eq(users.id, posts.authorId));

const query2 = db
  .select({ id: users.id, name: sql<string>`users.name` })
  .from(users)
  .where(eq(users.id, 2));

const query3 = db
  .select({ id: users.id, name: users.name })
  .from(users)
  .where(eq(users.id, 3));

const query4 = db
  .select({ id: users.id, name: sql<string>`users.name` })
  .from(users)
  .where(eq(users.id, 4))
  .leftJoin(posts, eq(users.id, posts.authorId));

const query5 = db
  .select({ id: users.id, name: users.name })
  .from(users)
  .where(eq(users.id, 5));

const query6 = db
  .select({ id: users.id, name: users.name })
  .from(users)
  .where(eq(users.id, 2));

All of the following are valid drizzle syntax:

const result1 = await union(
  query1,
  query2,
  intersect(query3, query4, query5),
  query6,
).orderBy(desc(users.id))

const result2 = await union(
  union(
    union(query1, query2),
    intersect(intersect(query3, query4), query5),
  ),
  query6,
)

const result3 = await query1
  .union(query2)
  .union(query3.intersect(query4).intersect(query5))
  .union(query6)

const result4 = await query1
  .union(query2)
  .union(({ intersect }) => intersect(query3, query4, query5))
  .union(query6)

And will all result in the same query (MySql and Postgres):
Keep in mind that the queries are mutable, so if those 4 queries are run one after the other, the results would be unexpected.

"(((select \"users\".\"id\", \"users\".\"name\" from \"users\" left join \"posts\" on \"users\".\"id\" = \"posts\".\"author_id\"
where \"users\".\"id\" = $1) union (select \"id\", users.name from \"users\" where \"users\".\"id\" = $2))
union (((select \"id\", \"name\" from \"users\" where \"users\".\"id\" = $3) intersect (select \"users\".\"id\", users.name
from \"users\" left join \"posts\" on \"users\".\"id\" = \"posts\".\"author_id\" where \"users\".\"id\" = $4))
intersect (select \"id\", \"name\" from \"users\" where \"users\".\"id\" = $5)))
union (select \"id\", \"name\" from \"users\" where \"users\".\"id\" = $6)",
params: [ 1, 2, 3, 4, 5, 2 ]

But in SQLite, parenthesis are not allowed, or order by / limit on the left of the set operator clause, so the resulting query will be:

"select \"users\".\"id\", \"users\".\"name\" from \"users\" left join \"posts\" on \"users\".\"id\" = \"posts\".\"author_id\"
where \"users\".\"id\" = ? union select \"id\", users.name from \"users\" where \"users\".\"id\" = ?
union select \"id\", \"name\" from \"users\" where \"users\".\"id\" = ? intersect select \"users\".\"id\", users.name
from \"users\" left join \"posts\" on \"users\".\"id\" = \"posts\".\" author_id\" where \"users\".\"id\" = ?
intersect select \"id\", \"name\" from \"users\" where \"users\".\"id\" = ? union select \"id\", \"name\"
from \"users\" where \"users\".\"id\" = ?",
params: [ 1, 2, 3, 4, 5, 2 ]

Selecting different columns names, with different types or different number of them will result in a type error that if ignored, will result in a runtime error as it will be invalid SQL syntax.

 - Added coments for type tests

 - Added runtime check that throws an error if the select have keys in different order

 - Added tests for the new error throwing behavior
Copy link
Contributor

@dankochetov dankochetov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the PR's destination to beta instead of main. There were some query builder type changes, so you will probably have some merge conflicts.

Also, you'd need to update the SetOperator classes according to the changes in other query builders.

drizzle-orm/src/mysql-core/dialect.ts Outdated Show resolved Hide resolved
drizzle-orm/src/mysql-core/query-builders/set-operators.ts Outdated Show resolved Hide resolved
drizzle-orm/src/mysql-core/query-builders/set-operators.ts Outdated Show resolved Hide resolved
drizzle-orm/src/mysql-core/query-builders/set-operators.ts Outdated Show resolved Hide resolved
drizzle-orm/src/mysql-core/dialect.ts Show resolved Hide resolved
@Angelelz Angelelz changed the base branch from main to beta October 5, 2023 21:40
@enfipy
Copy link

enfipy commented Oct 9, 2023

Just want to say that I absolutely love this feature and can't wait to see it released! 🔥

@Angelelz Angelelz requested a review from dankochetov October 15, 2023 02:11
@Angelelz
Copy link
Collaborator Author

The implementation is complete.
I included similar types as the rest of the query builders to avoid calling the same method twice.

@datner
Copy link

datner commented Oct 17, 2023

the hype is real, lets gooooooo

Copy link
Contributor

@dankochetov dankochetov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LET'S GO THE HYPE IS REAL

@dankochetov dankochetov merged commit 1a482ce into drizzle-team:beta Nov 2, 2023
4 checks passed
prometixX pushed a commit to prometixX/drizzle-orm-mssql that referenced this pull request Jan 28, 2024
@csimmons0 csimmons0 mentioned this pull request Sep 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add UNION support
5 participants