Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming uploads #47

Closed
kylebarron opened this issue Oct 24, 2024 · 4 comments · Fixed by #54
Closed

Streaming uploads #47

kylebarron opened this issue Oct 24, 2024 · 4 comments · Fixed by #54

Comments

@kylebarron
Copy link
Member

kylebarron commented Oct 24, 2024

We should allow async iterator input into put. Then we can pass the result of get directly into put and have streaming transfers from one store to another.

@machichima
Copy link

This feature would be extremely helpful! We're really looking forward to it. Is there any update on this issue?

@kylebarron
Copy link
Member Author

I attempted it in #54. It compiles fine, but then appeared to hang in my manual tests. I'm not sure what the issue is and haven't had time to look back into it. If you're interested, you can go to that branch and test it out (you can just pip install . from the obstore directory as long as you have Rust installed)

@kylebarron
Copy link
Member Author

It turned out that the only reason for my errors in #54 was that I was hitting my request timeout. Raising the timeout shows that the PR does in fact work.

@rupurt
Copy link

rupurt commented Jan 15, 2025

You absolute legend @kylebarron! If we ever meet in person I owe you a 6 pack 🍻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants