You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Davidslv
wants to merge
3
commits into
keolo:master
from
Davidslv:davidslv/add-stream-for-mixpanel-response
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Hey @keolo, I've modified the Mixpanel::URI.get to handle stream instead of parsing and read the whole response, which can be quite big on an export, since it's per day, a project with many events/properties or events with many properties can take quite a while to "download" everything into memory. This should close#44
Hey @keolo, I did it in a way that it didn't change the api, and still made your specs pass.
Although because of the latest, I think the implementation is not correct.
Meaning that now it downloads the file in chunks but then it still has to put it in a string variable, which will most likely cause memory issues. So I left your method alone and I created another one called download which accepts a directory and a filename so that all the writing is done in chunks to a specified file. You can check it out here: https://github.com/Superfling/mixpanel_client/blob/master/lib/mixpanel/uri.rb#L21 although we decided to not download the whole export for what we need, hence the file_size limit here ( https://github.com/Superfling/mixpanel_client/blob/master/lib/mixpanel/uri.rb#L30 ) but that can be changed easily if you decide to adopt it.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey @keolo, I've modified the
Mixpanel::URI.getto handle stream instead of parsing and read the whole response, which can be quite big on an export, since it's per day, a project with many events/properties or events with many properties can take quite a while to "download" everything into memory. This should close #44Thank you