Can't sync repos that have different paths.

[TracyMRohlin]

I have a copy of my repo in my databricks workspace: /Workspace/Users/[email protected]/foobar and a local copy located here: /Users/myname/Code_Repos/machine-learning/foo_bar. I'd like to be able to link the two folders so i can sync between them for development, but the issue is that when i try to set the paths and click download, the download will include the entire parent structure of my workspace on databricks.

E.g., say i have a local file located here:

/Users/myname/Code_Repos/machine-learning/foo_bar/src/pages.py

and also here on databricks:

/Workspace/Users/[email protected]/foobar/src/pages.py

I want to up/download them so that if i make changes locally i can sync and test in my workspace. But when i download, it now creates this folder structure:

src/Workspace/Users/[email protected]/foobar/src/

This is the user settings:

{
    "jupyter.interactiveWindow.cellMarker.codeRegex": "^# COMMAND ----------|^# Databricks notebook source|^(#\\s*%%|#\\s*\\<codecell\\>|#\\s*In\\[\\d*?\\]|#\\s*In\\[ \\])",
    "jupyter.interactiveWindow.cellMarker.default": "# COMMAND ----------",
    "editor.defaultFoldingRangeProvider": "ms-python.python",
    "editor.defaultFormatter": "ms-python.python",
    "notebook.defaultFormatter": "paiqo.databricks-vscode",
    "databricks.connection.default.displayName": "Default",
    "databricks.connection.default.apiRootUrl": <host>,
    "databricks.connection.default.personalAccessToken": <token>,
    "databricks.connection.default.localSyncSubfolders": {

        "Workspace": "src",
        "Clusters": "Clusters",
        "DBFS": "DBFS",
        "Jobs": "Jobs"
    },
    "databricks.connections": [
        {
            "apiRootUrl": <host>,
            "displayName": "us-01",
            "personalAccessTokenSecure": {
                "keyTarSettingName": "us-01-API-Token"
            }
        },
        "databricks.lastActiveConnection": "us-01",
        "databricks.azure.workspaces": [

        
        ]
}

[gbrueckl]

I just released v2.5.0 which now features a new config setting databricks.workspace.root to set the root of the workspace browser and also impacts the relative local paths for the sync

so I guess in your case you would set it to /Workspace/Users/[email protected]/foobar

let me know if it works for you!

[TracyMRohlin]

I can't test since I no longer see any connected workspaces:

[gbrueckl]

can you check your user and/or workspace settings please?
does the new config setting show up at all?!?
I just ran a test on my regular installation and it did not show up when I just updated the extension
after restarting VSCode it was showing up and the workspace browser also worked again

your are not using the extension in the browser (e.g. via https://vscode.dev), do you?
whats your OS?

[TracyMRohlin]

MAC OS 14.2.1 (23C71)
I'm using the extension via the actual app.

I updated and it also did not have the field either, I had to manually set it:

`

{
"folders": [
	{
		"name": "Databricks - Workspace",
		"uri": "wsfs:/"
	}
],
"settings": {
	"notebook.defaultFormatter": "ms-python.python",
	"databricks.connection.default.displayName": "Default",
	"databricks.connection.default.apiRootUrl": "<host>",
	"databricks.connection.default.localSyncSubfolders": {
		"Workspace": "Workspace",
		"Clusters": "Clusters",
		"DBFS": "DBFS",
		"Jobs": "Jobs"
	},
	"databricks.connections": [
		{
			"apiRootUrl": "<host>",
			"displayName": "us-01",
			"localSyncFolder": "/Users/email/Code_Repos/machine-learning/foo_bar",
			"personalAccessTokenSecure": {
				"keyTarSettingName": "token"
			}
		}
	],
	"databricks.lastActiveConnection": "us-01",
	"databricks.azure.workspaces": [
		
	],
	"databricks.connectionManager": "Databricks CLI Profiles",
	"workbench.editor.defaultBinaryEditor": "default",
	"databricks.connection.default.exportFormats": {

		"Scala": ".scala",
		"Python": ".ipynb",
		"SQL": ".sql",
		"R": ".r"
	},
	"databricks.connection.default.localSyncFolder": "/Users/email/Code_Repos/machine-learning/foo_bar"",
	"databricks.connection.default.personalAccessTokenSecure": {
		"keyTarSettingName": "Default-API-Token"
	}
}

}

`

[gbrueckl]

so do you see the new setting?

[TracyMRohlin]

Oh yes i do see it now. now the issue seems to be it is downloading at /Users/email/Databricks-VSCode/US01/Workspace/src/table_dependencies/ and not at /Users/email/Code_Repos/machine-learning/data_quality/src/table_dependencies/

[gbrueckl]

basically everything under databricks.workspace.root will be downloaded to <localSyncFolder>/<localSyncSubfolder for workspace> so I think this should just be a configuration issue on your side

I guess you need to change the localSyncFolder settings for this

[TracyMRohlin]

i have changed my local sync subfolders/parent folders. the issue is that the root seems to be hardcoded at /Users/email/Databricks-VSCode/US01/ . I grepped for this in the settings.json and it's no where in here.

[gbrueckl]

whats your setting for "databricks.connectionManager"?
the local sync folder is configured as part of the connection manager
the default is the user-directory if you do not specify anything else

in the screenshot above you were using Databricks CLI Profiles so please check out https://github.com/paiqo/Databricks-VSCode?tab=readme-ov-file#additional-settings