Skip to content

Conversation

vinay79n
Copy link
Contributor

@vinay79n vinay79n commented Sep 23, 2025

Added select_dev_query_tags() function to attach query tags and comments for cost tracking in Snowflake queries, in the publish(), publish_pandas(), and query_pandas_from_snowflake() functions.

The tags are fetched in this fashion which follows the suggested query comment/tag structure (recommended by Select Dev team), which will get added as SQL query comment in the queries that go through publish() and query_pandas_from_snowflake() and as session query tag in case of publish_pandas() (as publish_pandas() internally uses snowflake's write_pandas() and dirctly adding SQL query comment when doing write_pandas() is not possible, so session query tag will be added in the time of query_pandas_from_snowflake():

Screenshot 2025-09-23 at 5 10 40 PM Screenshot 2025-09-23 at 5 11 28 PM

…cking in Snowflake queries , in the publish(), publish_pandas(), and query_pandas_from_snowflake() functions.
@wiz-55ccc8b716
Copy link

wiz-55ccc8b716 bot commented Sep 23, 2025

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
SAST Finding SAST Findings 5 Medium
Total 5 Medium

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements cost tracking functionality for Snowflake queries by adding a get_tags() function that extracts metadata from Metaflow flow runs and attaches it as query comments or session tags.

  • Added get_tags() function to extract flow metadata for cost tracking
  • Modified query functions to include cost tracking tags in SQL comments
  • Added session query tags for pandas operations where SQL comments aren't supported

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
src/ds_platform_utils/metaflow/write_audit_publish.py Added get_tags() function and modified publish() to include query comments for cost tracking
src/ds_platform_utils/metaflow/pandas.py Modified publish_pandas() and query_pandas_from_snowflake() to add cost tracking via session tags and query comments

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

… rename get_tags to select_dev_query_tags, add a descriptive docstring, improve publish_pandas comment for query tagging, warn when tags are missing to encourage adding them to flows, and other minor tweaks.
@vinay79n vinay79n requested a review from phitoduck September 29, 2025 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants