feat: Switch to JSONField for SDK metrics labels
#5833
+96
−11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 3 Skipped Deployments
|
github-actions
bot
added
api
Docker builds report
|
github-actions
bot
added
feature
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #5833 +/- ##
=======================================
Coverage 97.82% 97.82%
=======================================
Files 1258 1259 +1
Lines 44745 44747 +2
=======================================
+ Hits 43771 43773 +2
Misses 974 974 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Confirming this fix allows CockroachDB to work without needing to update Psycopg to 3.1+. |
Zaimwa9
requested changes
Jul 28, 2025
|
@Zaimwa9 Indeed these migrations look very similar, but their purpose is actually different:
So, to answer your question, no, we can't merge these migrations as they cater for different scenarios, and I argue they are not in fact redundant. |
Zaimwa9
previously approved these changes
Jul 28, 2025
emyller
requested changes
Jul 28, 2025
Comment on lines
12
to
32
| operations = [ | ||
| migrations.AlterField( | ||
| model_name="apiusagebucket", | ||
| name="labels", | ||
| field=models.JSONField(default=dict), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name="apiusageraw", | ||
| name="labels", | ||
| field=models.JSONField(default=dict), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name="featureevaluationbucket", | ||
| name="labels", | ||
| field=models.JSONField(default=dict), | ||
| ), | ||
| migrations.AlterField( | ||
| model_name="featureevaluationraw", | ||
| name="labels", | ||
| field=models.JSONField(default=dict), | ||
| ), |
There was a problem hiding this comment.
To my understanding, this will lead to multiple alter column ... type statements. Is casting hstore to json as simple as that in, at least, Postgres?
I'm also curious as to the impact of this to existing installs.
There was a problem hiding this comment.
Good catch, we actually need a USING clause. Added in 091006e.
|
I forgot to add as notes to my last review: not actually requesting any changes unless proven necessary by the comment I added. |
github-actions
bot
added
feature
emyller
reviewed
Aug 8, 2025
Comment on lines
+37
to
+70
| migrations.RunSQL( | ||
| # Only alter the columns that are currently hstore. | ||
| # See 0006_add_labels to understand why we are doing this. | ||
| sql=""" | ||
| DO $$ | ||
| BEGIN | ||
| FOR relname IN | ||
| SELECT c.relname | ||
| FROM pg_class c | ||
| JOIN pg_attribute a ON a.attrelid = c.oid | ||
| JOIN pg_type t ON a.atttypid = t.oid | ||
| WHERE c.relname IN ( | ||
| 'app_analytics_apiusagebucket', | ||
| 'app_analytics_apiusageraw', | ||
| 'app_analytics_featureevaluationbucket', | ||
| 'app_analytics_featureevaluationraw' | ||
| ) | ||
| AND a.attname = 'labels' | ||
| AND t.typname = 'hstore' | ||
| LOOP | ||
| EXECUTE format( | ||
| 'ALTER TABLE %I | ||
| ALTER COLUMN labels TYPE jsonb USING hstore_to_json(labels), | ||
| ALTER COLUMN labels SET DEFAULT ''{}''::jsonb', | ||
| relname | ||
| ); | ||
| END LOOP; | ||
| END | ||
| $$; | ||
| """, | ||
| # We don't want hstore in the database at all, | ||
| # so don't do anything for reverse SQL. | ||
| reverse_sql="", | ||
| ), |
There was a problem hiding this comment.
The SQL code looks fine, though I think we should benefit from the ORM abstraction until we need to fall back to raw SQL.
Here's an untested brain dump.
Given:
def upgrade_field(model_name: str, field_name: str) -> Callable[[Apps, SchemaEditor], None]:
def upgrade(apps: Apps, schema_editor: SchemaEditor) -> None:
model = apps.get_model("app_analytics", model_name)
field = model._meta.get_field(field_name)
if isinstance(field, models.JSONField):
continue
schema_editor.execute(f"""
ALTER TABLE {model._meta.db_table}
ALTER COLUMN {field.column} TYPE jsonb USING hstore_to_json({field.column}),
ALTER COLUMN {field.column} SET DEFAULT '{{}}'::jsonb
""")
return upgradeSuggested replacement for the SQL script:
Suggested change
| migrations.RunSQL( | |
| # Only alter the columns that are currently hstore. | |
| # See 0006_add_labels to understand why we are doing this. | |
| sql=""" | |
| DO $$ | |
| BEGIN | |
| FOR relname IN | |
| SELECT c.relname | |
| FROM pg_class c | |
| JOIN pg_attribute a ON a.attrelid = c.oid | |
| JOIN pg_type t ON a.atttypid = t.oid | |
| WHERE c.relname IN ( | |
| 'app_analytics_apiusagebucket', | |
| 'app_analytics_apiusageraw', | |
| 'app_analytics_featureevaluationbucket', | |
| 'app_analytics_featureevaluationraw' | |
| ) | |
| AND a.attname = 'labels' | |
| AND t.typname = 'hstore' | |
| LOOP | |
| EXECUTE format( | |
| 'ALTER TABLE %I | |
| ALTER COLUMN labels TYPE jsonb USING hstore_to_json(labels), | |
| ALTER COLUMN labels SET DEFAULT ''{}''::jsonb', | |
| relname | |
| ); | |
| END LOOP; | |
| END | |
| $$; | |
| """, | |
| # We don't want hstore in the database at all, | |
| # so don't do anything for reverse SQL. | |
| reverse_sql="", | |
| ), | |
| migrations.RunPython( | |
| code=upgrade_field("apiusagebucket", "labels"), | |
| reverse_code=migrations.RunPython.noop, | |
| ), | |
| migrations.RunPython( | |
| code=upgrade_field("apiusageraw", "labels"), | |
| reverse_code=migrations.RunPython.noop, | |
| ), | |
| migrations.RunPython( | |
| code=upgrade_field("featureevaluationbucket", "labels"), | |
| reverse_code=migrations.RunPython.noop, | |
| ), | |
| migrations.RunPython( | |
| code=upgrade_field("featureevaluationraw", "labels"), | |
| reverse_code=migrations.RunPython.noop, | |
| ), |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks for submitting a PR! Please check the boxes below:
docs/if required so people know about the feature!Changes
This migrates all the
labelsfields added in #5623 tojsonbas thehstoresupport turned out to be problematic:hstoreextension requires enabling additional parameter sets in some cloud environments (e.g. Azure).Note that due to problem 1, I've removed the
CREATE EXTENSIONoperation from the original migration.How did you test this code?
Migrations and tests run ok.
Will be smoke-tested against CockroachDB as well.