[Databricks E2E] Sprint 5 improvements #1082
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#988) * automate region vm + dbx type cluster in the region * Update e2e_samples/parking_sensors/scripts/configure_databricks.sh Co-authored-by: Elena Terenzi <[email protected]> * filtering photon support --------- Co-authored-by: Elena Terenzi <[email protected]>
* fix breaking change from dbx cli output * fix hyperlink check
Adding the Cleanup for federal credentials which follows a certain order: Get the SP Objc ID of the Service Connection Delete federated credentials from the SP Object ID Delete the service connection
Service Connections are using the Workload Identity which comes from a configuration file. The cleanup in case of re-start of this script also follows a peculiar order: Get SP object if according to the Service Connection Get the federate credential according to the SP Delete federated credentials Delete the Service Connection
changing the docs with the required permission
rephrase the permissions needed for azdevops project
Co-authored-by: Yennifer Santos <[email protected]>
…tions_azure.sh Co-authored-by: Yennifer Santos <[email protected]>
I adjust the order of the permissions
adding know issues how to delete federate credentials
identation was not correct
adding the following changes: 1) Function for cleanup 2)Function for sleep 3)Remove log file
Implement function for cleanup and function for sleep
Added cleanup_federate_credentials and wait_for_credentials used in the clean_up.sh and deploy_azdo_service_connection.sh
rename wait_for_cleanup to wait_for_process
renamed wait_for_cleanup to wait_for_process
typo: comtain to contain fxed
added Elena's suggestion of rewrite Co-authored-by: Elena Terenzi <[email protected]>
remove the log for SP
Clean up of the log suppress update pipelines message remove response from logging
changing the log for federate credentials removal
adding the SP in the log message
checking if the file exist before removal.
Co-authored-by: Elena Terenzi <[email protected]>
Co-authored-by: Elena Terenzi <[email protected]>
Co-authored-by: Elena Terenzi <[email protected]>
…tions_azure.sh Co-authored-by: Elena Terenzi <[email protected]>
…tions_azure.sh Co-authored-by: Elena Terenzi <[email protected]>
Co-authored-by: Elena Terenzi <[email protected]>
Co-authored-by: Elena Terenzi <[email protected]>
Co-authored-by: Elena Terenzi <[email protected]>
role unecessary
adding -0 none to resolve conflict
…parkingsensor_v1_lilem
…sor_v1_lilem E2e/databricks/parkingsensor v1 lilem
Fix dashboard bicep Warnings #955
…val (#1047) * remove federated credentials only when needed * fix typo
* remove SPN for AzDO service connection * minor edits * minor edits * minor typo * add readme change for manual deletion
* Deploy.sh will have the option to deploy as: 1) dev 2) Dev and Stage 3) Dev, Stage and Prod Common.sh will heold the function for this changes envtemplate the new environmnent variable for the deployment options deploy_azdo_pipelines.sh will deploy the release pipeline if option 2 or 3 are choosen which implies the deploy of t least 2 environmnents * adjusting the readme according to the latest version in v1 * Seems this session was not veru well documented - **Workaround - Manually Deleting Federated Credentials:** * to* typo * Update e2e_samples/parking_sensors/README.md Co-authored-by: Copilot <[email protected]> * Remove duplicate deployment instruction in README fixing copilot fix * Adding if for deploy.sh respect configuration in the enviromnent variable Initialize env variable in the init_environment * adjust the readme for env. varaible configuration * Clarify deployment options and environment variable usage Adjusting explanations about env variable in the readme and envtemplate * Update workaround for deleting Azure DevOps Service Connection fixing conflict * Refactor cleanup function to delete service principal fix the conflict --------- Co-authored-by: Copilot <[email protected]>
* Recreates the CI CD process diagrams in drawio * Remove the ADF publish step from CI CD process diagrams * Update the README.md to reflect the changes in the process sequence and diagram files --------- Co-authored-by: Jose Perales <[email protected]>
* Fix devcontainer for wheel library builds * env file required for devcontainer * Update e2e_samples/parking_sensors/src/ddo_transform/.devcontainer/Dockerfile
* fix: release paths and cluster id * fix: adding azdo file * fix: removing unnecessary config * update: readMe, optimize code flow * update: upgrade DB CLI and fix release pipeline * fix: pr comments
* Add PowerShell and Bash scripts for Synapse Pause/Resume * Support Manual and Automated Synapse Pause PowerShell Script with Additional features * Add README file for Synapse Pause script * Add References section to Synapse Pause README file * Add InstallModules parameter to pause_resume_synapse script for automatic module installation * Enhance README and script usage documentation for pause_resume_synapse and improved logging for state transitions * Update README_pause_resume_synapse to include example values for parameters * Update README_pause_resume_synapse to provide specific examples for ResourceGroups parameter
LiliamLeme
approved these changes
Feb 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Type of PR
Purpose
Does this introduce a breaking change? If yes, details on what can break
Author pre-publish checklist
Validation steps
Issues Closed or Referenced