Add crypto_data_pipeline Package to dbt Package Hub

[1997mahadi]

Description

This PR adds the crypto_data_pipeline package to the dbt Package Hub. The package facilitates the ingestion and processing of cryptocurrency data using DLT (Data Loading Tool) and stores the data in DuckDB. It includes functionalities to fetch historical crypto data, generate metadata, and simulate purchase data. This project aims to provide a robust solution for crypto data analysis with an emphasis on ease of use and customizability.

Link to your package's repository: crypto_data_pipeline

Checklist

First run experience

• (Required): The package includes a license file detectable by GitHub, such as the Apache 2.0 or MIT license.
• The package includes a README which explains how to get started with the package and customize its behavior.
• The README indicates which data warehouses/platforms are expected to work with this package.

Customizability

• The package uses ref or source, instead of hard-coding table references.
• Provides a mechanism (such as variables) to customize the location of source tables.
• Does not assume database/schema names in sources.

Dependencies

• The package has set a supported require-dbt-version range in dbt_project.yml.
• Dependencies are imported from the dbt Package Hub when available, as opposed to a git installation.
• Dependencies contain the widest possible range of supported versions, to minimize issues in dependency resolution.
• In particular, dependencies are not pinned to a patch version unless there is a known incompatibility.

Interoperability

• The package does not override dbt Core behavior in such a way as to impact other dbt resources (models, tests, etc.) not provided by the package.
• The package uses the cross-database macros built into dbt Core where available.
• The package disambiguates its resource names to avoid clashes with nodes that are likely to already exist in a project.

Versioning

• (Required): The package's git tags validate against the regex defined in version.py.
• The package's version follows the guidance of Semantic Versioning 2.0.0.

[1997mahadi]

@dbeatty10 @joellabes Could you please review this PR? Thanks!

[joellabes]

@1997mahadi please add a licence file and then we can get this merged!

[1997mahadi]

Hi @joellabes,

I have added the MIT license file to the repository. Please review the changes and let me know if there's anything else needed.

Thank you!

[1997mahadi]

Hi @joellabes,

I have added the MIT license file to the repository. You can review the changes here.

Thank you

[joellabes]

thank you!

[1997mahadi]

Hi @joellabes, my package dbt-dlt-ingestion-pipeline was merged about 24 hours ago, but it hasn’t appeared on the dbt Hub yet. Could you please check if everything is in order? Thank you!

[joellabes]

Your version number isn't being picked up, because it's missing a patch version: https://regex101.com/r/ZXbClh/1 (that’s what (Required): The package's git tags validate against the regex defined in version.py is for)

If you make a version that is called v1.0.0 or 1.0.0 you should be ok

[1997mahadi]

@joellabes, I've added the version tag v1.0.0 as suggested. Could you please check if everything is now in order for the package to be listed on dbt Hub? Thank you!

[joellabes]

The hub checks for new packages once an hour - it should show up shortly!

[1997mahadi]

@joellabes,

I've made the necessary updates to my repository, including creating a new tag v1.0.0 for my package crypto_data_pipeline. It's been a few hours since the updates, and my package still hasn't been listed on the dbt Hub. Could you please check if there's anything else that needs to be done or if there's an issue with the listing process?

Thanks for your assistance!

Best regards,
Mahadi

[joellabes]

The v1.0.0 tag doesn't have a dbt_project.yml file at its root: https://github.com/1997mahadi/dbt-dlt-ingestion-pipeline/tree/v1.0.0

You'll need to structure your project like this other dlt project with the dbt_project.yml at the root and pointing at subdirectories for model-paths etc https://github.com/dlt-hub/dbt_ga4/blob/main/dbt_project.yml

[joellabes]

(and then you'll need to delete the old v1.0.0 tag and do a new release, or it will get stuck trying to handle the old one)