-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
✨Add support for local source files #156
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good! Could you add a small test for it?
Just letting you know that we are working on a better process than modifying the |
Will add a test probably on Monday! Thanks for letting me know, sounds great. |
Signed-off-by: zethson <[email protected]>
Signed-off-by: zethson <[email protected]>
Signed-off-by: zethson <[email protected]>
Signed-off-by: zethson <[email protected]>
I'm sorry for abusing this PR to ensure that the CI now runs fine on forked PRs but it does now. So with your test on Monday, we can merge this although this design is going to be succeeded in the future. Thank you very much! |
Have now added the test! thanks |
Signed-off-by: zethson <[email protected]>
Signed-off-by: zethson <[email protected]>
Thank you so much! I made a few edits myself to save you a bit of time and get this in more quickly. I hope you don't mind. If you'd rather have me review smaller PRs instead of making changing to them, please let me know!
|
No that's perfect, thanks so much for making those improvements. Completely understood re pre-commit and tempfile! |
This is a very small PR and I'm definitely open to better ways of doing this!
Some use-cases, e.g. debugging, may require local files to be loaded as a source.
As an example, I added to
sources_local.yaml
the following lines:With the change in this PR I can load this local parquet file. There's almost certainly a better way of doing this so please do enlighten me!
Thanks