Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run another notebook from the current one #93

Closed
nikjuice opened this issue Sep 14, 2022 · 20 comments
Closed

Run another notebook from the current one #93

nikjuice opened this issue Sep 14, 2022 · 20 comments

Comments

@nikjuice
Copy link

I have faced with an issue running the another notebook. So in the beginning of the our notebook we have something like
%run ../shared/data_platform_globals
When I run this using the extension I got following:
Exception: File '..shared/data_platform_globals.py' not found.

Running this from Databricks works without an issue.
Please let me know if more details needed.

@gbrueckl
Copy link
Collaborator

gbrueckl commented Sep 14, 2022 via email

@nikjuice
Copy link
Author

Thank you for quick update!

@yrossi55tf1
Copy link

Hello @gbrueckl , any other method instead of %run to do the same ?

@gbrueckl
Copy link
Collaborator

Not as of now, you could only copy&paste the referenced code directly into your notebook

I am afraid you need to wait till this feature is added which will hopefully be in the next few weeks

@gbrueckl
Copy link
Collaborator

just published v1.1.3 and %run should now work!

a known limitation is that it currently only works if the referenced file uses the same language as the notebook (Python/PySpark)

let me know your feedback!

@yrossi55tf1
Copy link

yrossi55tf1 commented Sep 19, 2022

@gbrueckl Wonderful amazing work in quick time ! Just I'm getting a new error now related to the path :

EntryNotFound (FileSystemError): Impossible de résoudre le fichier inexistant « /Users/yrossi/Documents/Databricks/Workspace/data-analytics/cockpit/'/Users/yrossi/Documents/Databricks/Workspace/data-analytics/utils »

@gbrueckl
Copy link
Collaborator

well, you also need to download the referenced notebook locally
so it will always execute the the local file referenced by the relative path passed to %run

assuming you use %run ./util you need to make sure that util.ipynb exists in the same folder as the notebook you are currently executing

@yrossi55tf1
Copy link

Ok thanks, so it has to be in the same local folder ! Thanks for explaining

@gbrueckl
Copy link
Collaborator

could you also post your full %run command please
the error message makes me think you use singe quotes around the referenced file, is that true?
in case you have blanks or any special characters in the referenced file name, you should use double quotes
(single quotes also do not work on Databricks)

@gbrueckl
Copy link
Collaborator

I also just added support for dbutils.notebook.run in v1.1.4 when running in Live-mode using a dbws:/ mount in VSCode workspace!

@gbrueckl
Copy link
Collaborator

gbrueckl commented Oct 3, 2022

closing this as it was implemented and got no negative feedback anymore

@gbrueckl gbrueckl closed this as completed Oct 3, 2022
@nikjuice
Copy link
Author

nikjuice commented Oct 4, 2022

Sorry for the late feedback, it works. Thanks a lot for so quick adding.

@theiotidiot
Copy link

This doesn't appear to function correctly for me. When attempting to run a cmd with %run magic cmd - I get a file system error. EntryNotFound (FileSystemError) : Unable to resolve nonexistent file 'c:\Users\me\databricks-project\my-current-location\util\generic_notebook'.

Its checking for a notebook relative to the notebook's current location, but it should be relative to the workspace. i.e. 'c:\Users\me\databricks-project\util\generic_notebook'.

@gbrueckl
Copy link
Collaborator

Very much depends on how you %run command looks
Can you please share the code and also the file/folder structure

Relative paths shoukd work but i could think of an issue with absolute paths

@theiotidiot
Copy link

theiotidiot commented Oct 13, 2022

Code : (calling_notebook)
cmd1 : %run databricks-project/util/generic_notebook
cmd2 : doing something with variables from the nested notebook in cmd1

Folder Structure :

databricks-project
└─ util
     └─ generic_notebook
└─ calling_notebook

@gbrueckl
Copy link
Collaborator

gbrueckl commented Oct 13, 2022 via email

@theiotidiot
Copy link

I downloaded the notebooks in question, from the Databricks tab, and it added a 'Workspace' folder to my VSCode project folder. I am opening the local notebook copy and running that notebook on our databricks cluster. When I use '%run /path/to/notebook' it does a relative search for the notebook I'm attempting to run, but to match the functionality of the Databricks editor, it should be an absolute path from the 'Workspace' folder.

If I was to use a relative path it would make things more difficult. We have a ton of notebooks in various folder structures and if we had to use relative paths for each one of those notebooks, it would be a major headache.

@gbrueckl
Copy link
Collaborator

gbrueckl commented Oct 17, 2022

I think I could fix it - the new version will probably be released this week so stay tuned!

reopening this ticket to update you here

@gbrueckl gbrueckl reopened this Oct 17, 2022
@gbrueckl
Copy link
Collaborator

the issue should be fixed in this release, can you please try it and report back - thanks!
https://github.com/paiqo/Databricks-VSCode/releases/tag/v1.2.0

Note: it is not yet published to the VSCode gallery

@gbrueckl
Copy link
Collaborator

just published v1.2.1 to the VSCode gallery
closing as fixed as I did not get any negative feedback on the preivous fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants