Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate why dbt test command fails on dbt duckdb #91

Open
anna-geller opened this issue Feb 19, 2024 · 1 comment
Open

Investigate why dbt test command fails on dbt duckdb #91

anna-geller opened this issue Feb 19, 2024 · 1 comment
Labels
bug Something isn't working
Milestone

Comments

@anna-geller
Copy link
Member

Feature description

id: dbt_duckdb
namespace: blueprint

tasks:
  - id: dbt
    type: io.kestra.core.tasks.flows.WorkingDirectory
    tasks:
    - id: cloneRepository
      type: io.kestra.plugin.git.Clone
      url: https://github.com/kestra-io/dbt-example
      branch: main

    - id: dbt-build
      type: io.kestra.plugin.dbt.cli.DbtCLI
      runner: DOCKER
      docker:
        image: ghcr.io/kestra-io/dbt-duckdb:latest
      commands:
        - dbt deps
        - dbt build
        - dbt test # remove this line and everything will work just fine
      profiles: |
        my_dbt_project:
          outputs:
            dev:
              type: duckdb
              path: ":memory:"
              fixed_retries: 1
              threads: 1
              timeout_seconds: 300
          target: dev

this will fail with:

Runtime Error in test unique_my_second_dbt_model_id (models/example/schema.yml)
2024-02-19 12:43:11.523  Catalog Error: Table with name my_second_dbt_model does not exist!
2024-02-19 12:43:11.527  Did you mean "temp.pg_catalog.pg_enum"?
2024-02-19 12:43:11.528  LINE 15: from "memory"."main"."my_second_dbt_model"

Removing dbt test works.

Funny enough, dbt build already runs dbt test under the hood and it works just fine

@anna-geller anna-geller added the bug Something isn't working label Feb 19, 2024
@anna-geller
Copy link
Member Author

the same happens with earlier versions of dbt-duckdb:

id: dbt_custom_dependencies
namespace: dev
inputs:
  - id: dbt_version
    type: STRING
    defaults: "dbt-duckdb==1.6.0"
tasks:
  - id: git
    type: io.kestra.core.tasks.flows.WorkingDirectory
    tasks:
      - id: clone_repository
        type: io.kestra.plugin.git.Clone
        url: https://github.com/kestra-io/dbt-example
        branch: main
      - id: dbt
        type: io.kestra.plugin.dbt.cli.DbtCLI
        runner: DOCKER
        docker:
          image: python:3.11-slim
        beforeCommands:
          - pip install uv
          - uv venv --quiet
          - . .venv/bin/activate --quiet
          - uv pip install --quiet {{ inputs.dbt_version }}
        commands:
          - dbt deps
          - dbt build
          - dbt test
        profiles: |
          my_dbt_project:
            outputs:
              dev:
                type: duckdb
                path: ":memory:"
                fixed_retries: 1
                threads: 16
                timeout_seconds: 300
            target: dev

@anna-geller anna-geller added this to the v0.16.0 milestone Feb 19, 2024
@anna-geller anna-geller removed this from the v0.16.0 milestone Mar 19, 2024
@anna-geller anna-geller added this to the v0.22.0 milestone Apr 17, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Issues Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: Backlog
Development

No branches or pull requests

1 participant