Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent multiple SQL statements in the Query tasks #377

Open
3 tasks
anna-geller opened this issue Sep 3, 2024 · 2 comments
Open
3 tasks

Prevent multiple SQL statements in the Query tasks #377

anna-geller opened this issue Sep 3, 2024 · 2 comments
Assignees
Labels
area/plugin Plugin-related issue or feature request enhancement New feature or request kind/breaking-change Code modifications needed before upgrading

Comments

@anna-geller
Copy link
Member

Feature description

We currently have some inconsistencies, resulting in some plugins supporting multiple SQL statements (more of a side effect rather than intended behavior) while others support only a single SQL statement.

To simplify + add more consistency, we'll follow the approach:

  • Query task = run a single SQL statement
  • Queries task = run multiple SQL statements

We should prevent running multiple SQL queries in Query tasks:

  • backend: raise an error if multiple queries
  • frontend: ideally, warn in the editor if multiple queries are added (if not too complex to parse SQL in the Editor -- nice to have)
  • docs: adjust plugin docs to reflect that
@anna-geller anna-geller added enhancement New feature or request area/plugin Plugin-related issue or feature request labels Sep 3, 2024
@kestrabot kestrabot bot added this to Issues Sep 3, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Issues Sep 3, 2024
@Ben8t
Copy link
Member

Ben8t commented Sep 3, 2024

Have to try again, but running many SQL statements separated by ; should work 🤔 ?

Currently these queries run successfully with postgres.Query for example:

      CREATE SCHEMA IF NOT EXISTS weather;
      CREATE TABLE IF NOT EXISTS weather.tmp (
        "time" VARCHAR,
        "temperature_2m" VARCHAR,
        "latitude" VARCHAR,
        "longitude" VARCHAR,
        "city" VARCHAR
      );

      CREATE TABLE IF NOT EXISTS weather.temperature (
        "time" VARCHAR,
        "temperature_2m" VARCHAR,
        "latitude" VARCHAR,
        "longitude" VARCHAR,
        "city" VARCHAR,
        primary key(time, city)
      );

@Ben8t
Copy link
Member

Ben8t commented Sep 3, 2024

I can definitely see a need to run many queries in one task. Especially DDL queries which are often stored in one .sql file

- id: ddl
  type: postgres.Query
  sql : {{ read('ddl.sql') }} # where ddl.sql contain many SQL statements

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugin Plugin-related issue or feature request enhancement New feature or request kind/breaking-change Code modifications needed before upgrading
Projects
Status: Backlog
Development

No branches or pull requests

3 participants