-
Notifications
You must be signed in to change notification settings - Fork 0
feat: wrote get_tags() function #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…cking in Snowflake queries , in the publish(), publish_pandas(), and query_pandas_from_snowflake() functions.
Wiz Scan Summary
To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements cost tracking functionality for Snowflake queries by adding a get_tags()
function that extracts metadata from Metaflow flow runs and attaches it as query comments or session tags.
- Added
get_tags()
function to extract flow metadata for cost tracking - Modified query functions to include cost tracking tags in SQL comments
- Added session query tags for pandas operations where SQL comments aren't supported
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
File | Description |
---|---|
src/ds_platform_utils/metaflow/write_audit_publish.py | Added get_tags() function and modified publish() to include query comments for cost tracking |
src/ds_platform_utils/metaflow/pandas.py | Modified publish_pandas() and query_pandas_from_snowflake() to add cost tracking via session tags and query comments |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
… rename get_tags to select_dev_query_tags, add a descriptive docstring, improve publish_pandas comment for query tagging, warn when tags are missing to encourage adding them to flows, and other minor tweaks.
…derstanding and update references
…or enhanced context in the fetched tags.
…project), so it doesn’t depend on user tags.
Added
select_dev_query_tags()
function to attach query tags and comments for cost tracking in Snowflake queries, in thepublish()
,publish_pandas()
, andquery_pandas_from_snowflake()
functions.The tags are fetched in this fashion which follows the suggested query comment/tag structure (recommended by Select Dev team), which will get added as SQL query comment in the queries that go through
publish()
andquery_pandas_from_snowflake()
and as session query tag in case ofpublish_pandas()
(aspublish_pandas()
internally uses snowflake'swrite_pandas()
and dirctly adding SQL query comment when doingwrite_pandas()
is not possible, so session query tag will be added in the time ofquery_pandas_from_snowflake()
: