Restrict regions for GCP Cloud Build support #241
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Hi @martenlindblad , have you fully deploy the solution from scratch and tested the activation deployment to make sure your restrictions are in-place? Pls, provide a screenshot with evidence. Thank you! |
Of course!
After I switch to europe-west1 is succeeds. |
chmstimoteo
approved these changes
Nov 14, 2024
There was a problem hiding this comment.
LGTM! Thanks @martenlindblad for your collaboration.
Many more to come! ⭐
chmstimoteo
added a commit
that referenced
this pull request
Nov 15, 2024
* Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 15, 2024
* predicting for only the users with traffic in the past 72h - purchase propensity * running inference only for users events in the past 72h * including 72h users for all models predictions * considering null values in TabWorkflow models * deleting unused pipfile * upgrading lib versions * implementing reporting preprocessing as a new pipeline * adding more code documentation * adding important information on the main README.md and DEVELOPMENT.md * adding schedule run name and more code documentation * implementing a new scheduler using the vertex ai sdk & adding user_id to procedures for consistency * adding more code documentation * adding code doc to the python custom component * adding more code documentation * fixing aggregated predictions query * removing unnecessary resources from deployment * Writing MDS guide * adding the MDS developer and troubleshooting documentation * fixing deployment for activation pipelines and gemini dataset * Update README.md * Update README.md * Update README.md * Update README.md * removing deprecated api * fixing purchase propensity pipelines names * adding extra condition for when there is not enough data for the window interval to be applied on backfill procedures * adding more instructions for post deployment and fixing issues when GA4 export was configured for less than 10 days * removing unnecessary comments * adding the number of past days to process in the variables files * adding comment about combining data from different ga4 export datasets to data store * fixing small issues with feature engineering and ml pipelines * fixing hyper parameter tuning for kmeans modeling * fixing optuna parameters * adding cloud shell image * fixing the list of all possible users in the propensity training preparation tables * additional guardrails for when there is not enough data * adding more documentation * adding more doc to feature store * add feature store documentation * adding ml pipelines docs * adding ml pipelines docs * adding more documentation * adding user agent client info * fixing scope of client info * fix * removing client_info from vertex components * fixing versioning of tf submodules * reconfiguring meta providers * fixing issue 187 * chore(deps): upgrade terraform providers and modules version * chore(deps): set the provider version * chore: formatting * fix: brand naming * fix: typo * fixing secrets issue * implementing secrets region as tf variable * implementing secrets region as tf variable * last changes requested by lgrangeau * documenting keys location better * implementing vpc peering network * Update README.md * Rebase Main into Multi-property (#243) * Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * supporting property id in the resources --------- Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]> Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 15, 2024
* Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * supporting property id in the resources --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> Co-authored-by: Carlos Timoteo <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 15, 2024
* Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload * Support property id in resources (#246) * predicting for only the users with traffic in the past 72h - purchase propensity * running inference only for users events in the past 72h * including 72h users for all models predictions * considering null values in TabWorkflow models * deleting unused pipfile * upgrading lib versions * implementing reporting preprocessing as a new pipeline * adding more code documentation * adding important information on the main README.md and DEVELOPMENT.md * adding schedule run name and more code documentation * implementing a new scheduler using the vertex ai sdk & adding user_id to procedures for consistency * adding more code documentation * adding code doc to the python custom component * adding more code documentation * fixing aggregated predictions query * removing unnecessary resources from deployment * Writing MDS guide * adding the MDS developer and troubleshooting documentation * fixing deployment for activation pipelines and gemini dataset * Update README.md * Update README.md * Update README.md * Update README.md * removing deprecated api * fixing purchase propensity pipelines names * adding extra condition for when there is not enough data for the window interval to be applied on backfill procedures * adding more instructions for post deployment and fixing issues when GA4 export was configured for less than 10 days * removing unnecessary comments * adding the number of past days to process in the variables files * adding comment about combining data from different ga4 export datasets to data store * fixing small issues with feature engineering and ml pipelines * fixing hyper parameter tuning for kmeans modeling * fixing optuna parameters * adding cloud shell image * fixing the list of all possible users in the propensity training preparation tables * additional guardrails for when there is not enough data * adding more documentation * adding more doc to feature store * add feature store documentation * adding ml pipelines docs * adding ml pipelines docs * adding more documentation * adding user agent client info * fixing scope of client info * fix * removing client_info from vertex components * fixing versioning of tf submodules * reconfiguring meta providers * fixing issue 187 * chore(deps): upgrade terraform providers and modules version * chore(deps): set the provider version * chore: formatting * fix: brand naming * fix: typo * fixing secrets issue * implementing secrets region as tf variable * implementing secrets region as tf variable * last changes requested by lgrangeau * documenting keys location better * implementing vpc peering network * Update README.md * Rebase Main into Multi-property (#243) * Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * supporting property id in the resources --------- Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]> Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * Update terraform-template.tfvars * Update setup.py --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 22, 2024
* predicting for only the users with traffic in the past 72h - purchase propensity * running inference only for users events in the past 72h * including 72h users for all models predictions * considering null values in TabWorkflow models * deleting unused pipfile * upgrading lib versions * implementing reporting preprocessing as a new pipeline * adding more code documentation * adding important information on the main README.md and DEVELOPMENT.md * adding schedule run name and more code documentation * implementing a new scheduler using the vertex ai sdk & adding user_id to procedures for consistency * adding more code documentation * adding code doc to the python custom component * adding more code documentation * fixing aggregated predictions query * removing unnecessary resources from deployment * Writing MDS guide * adding the MDS developer and troubleshooting documentation * fixing deployment for activation pipelines and gemini dataset * Update README.md * Update README.md * Update README.md * Update README.md * removing deprecated api * fixing purchase propensity pipelines names * adding extra condition for when there is not enough data for the window interval to be applied on backfill procedures * adding more instructions for post deployment and fixing issues when GA4 export was configured for less than 10 days * removing unnecessary comments * adding the number of past days to process in the variables files * adding comment about combining data from different ga4 export datasets to data store * fixing small issues with feature engineering and ml pipelines * fixing hyper parameter tuning for kmeans modeling * fixing optuna parameters * adding cloud shell image * fixing the list of all possible users in the propensity training preparation tables * additional guardrails for when there is not enough data * adding more documentation * adding more doc to feature store * add feature store documentation * adding ml pipelines docs * adding ml pipelines docs * adding more documentation * adding user agent client info * fixing scope of client info * fix * removing client_info from vertex components * fixing versioning of tf submodules * reconfiguring meta providers * fixing issue 187 * chore(deps): upgrade terraform providers and modules version * chore(deps): set the provider version * chore: formatting * fix: brand naming * fix: typo * fixing secrets issue * implementing secrets region as tf variable * implementing secrets region as tf variable * last changes requested by lgrangeau * documenting keys location better * implementing vpc peering network * Update README.md * Rebase Main into Multi-property (#243) * Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * supporting property id in the resources * fixing iam member roles issues --------- Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]> Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 22, 2024
* predicting for only the users with traffic in the past 72h - purchase propensity * running inference only for users events in the past 72h * including 72h users for all models predictions * considering null values in TabWorkflow models * deleting unused pipfile * upgrading lib versions * implementing reporting preprocessing as a new pipeline * adding more code documentation * adding important information on the main README.md and DEVELOPMENT.md * adding schedule run name and more code documentation * implementing a new scheduler using the vertex ai sdk & adding user_id to procedures for consistency * adding more code documentation * adding code doc to the python custom component * adding more code documentation * fixing aggregated predictions query * removing unnecessary resources from deployment * Writing MDS guide * adding the MDS developer and troubleshooting documentation * fixing deployment for activation pipelines and gemini dataset * Update README.md * Update README.md * Update README.md * Update README.md * removing deprecated api * fixing purchase propensity pipelines names * adding extra condition for when there is not enough data for the window interval to be applied on backfill procedures * adding more instructions for post deployment and fixing issues when GA4 export was configured for less than 10 days * removing unnecessary comments * adding the number of past days to process in the variables files * adding comment about combining data from different ga4 export datasets to data store * fixing small issues with feature engineering and ml pipelines * fixing hyper parameter tuning for kmeans modeling * fixing optuna parameters * adding cloud shell image * fixing the list of all possible users in the propensity training preparation tables * additional guardrails for when there is not enough data * adding more documentation * adding more doc to feature store * add feature store documentation * adding ml pipelines docs * adding ml pipelines docs * adding more documentation * adding user agent client info * fixing scope of client info * fix * removing client_info from vertex components * fixing versioning of tf submodules * reconfiguring meta providers * fixing issue 187 * chore(deps): upgrade terraform providers and modules version * chore(deps): set the provider version * chore: formatting * fix: brand naming * fix: typo * fixing secrets issue * implementing secrets region as tf variable * implementing secrets region as tf variable * last changes requested by lgrangeau * documenting keys location better * implementing vpc peering network * Update README.md * Rebase Main into Multi-property (#243) * Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * supporting property id in the resources * fixing iam member roles issues * fixing issue with service account iam resources --------- Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]> Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]>
chmstimoteo
added a commit
that referenced
this pull request
Nov 22, 2024
* predicting for only the users with traffic in the past 72h - purchase propensity * running inference only for users events in the past 72h * including 72h users for all models predictions * considering null values in TabWorkflow models * deleting unused pipfile * upgrading lib versions * implementing reporting preprocessing as a new pipeline * adding more code documentation * adding important information on the main README.md and DEVELOPMENT.md * adding schedule run name and more code documentation * implementing a new scheduler using the vertex ai sdk & adding user_id to procedures for consistency * adding more code documentation * adding code doc to the python custom component * adding more code documentation * fixing aggregated predictions query * removing unnecessary resources from deployment * Writing MDS guide * adding the MDS developer and troubleshooting documentation * fixing deployment for activation pipelines and gemini dataset * Update README.md * Update README.md * Update README.md * Update README.md * removing deprecated api * fixing purchase propensity pipelines names * adding extra condition for when there is not enough data for the window interval to be applied on backfill procedures * adding more instructions for post deployment and fixing issues when GA4 export was configured for less than 10 days * removing unnecessary comments * adding the number of past days to process in the variables files * adding comment about combining data from different ga4 export datasets to data store * fixing small issues with feature engineering and ml pipelines * fixing hyper parameter tuning for kmeans modeling * fixing optuna parameters * adding cloud shell image * fixing the list of all possible users in the propensity training preparation tables * additional guardrails for when there is not enough data * adding more documentation * adding more doc to feature store * add feature store documentation * adding ml pipelines docs * adding ml pipelines docs * adding more documentation * adding user agent client info * fixing scope of client info * fix * removing client_info from vertex components * fixing versioning of tf submodules * reconfiguring meta providers * fixing issue 187 * chore(deps): upgrade terraform providers and modules version * chore(deps): set the provider version * chore: formatting * fix: brand naming * fix: typo * fixing secrets issue * implementing secrets region as tf variable * implementing secrets region as tf variable * last changes requested by lgrangeau * documenting keys location better * implementing vpc peering network * Update README.md * Rebase Main into Multi-property (#243) * Update README.md * ensure the build bucket is created in the specified region (#230) * Update audience_segmentation_query_template.sqlx * Update auto_audience_segmentation_query_template.sqlx * Update churn_propensity_query_template.sqlx * Update cltv_query_template.sqlx * Update purchase_propensity_query_template.sqlx * Restrict regions for GCP Cloud Build support (#241) * Update README.md * Move to uv (#242) * add uv required project table segment in toml file * switch to uv in terraform deployment * switch to uv * remove poetry usage from terraform * format * remove poetry * Add files via upload --------- Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]> * supporting property id in the resources * fixing iam member roles issues * fixing issue with service account iam resources * fixing issue with connection between vertex and bq --------- Co-authored-by: Carlos Timoteo <[email protected]> Co-authored-by: Laurent Grangeau <[email protected]> Co-authored-by: Charlie Wang <[email protected]> Co-authored-by: Mårten Lindblad <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Added restriction to region since the Cloud Build need to run in restriction free regions. The list of unrestricted regions are here: https://cloud.google.com/build/docs/locations
The container images are not built if not run from an unrestricted region, with a silent error in one of the local-exec resources.
How has this been tested?
I did one successful plan with europe-west1 as region, and an expected failed plan with europe-west3 as region, set in terraform.tfvars.
Checklist
Pipeline run links: