Remote first approach to take benchmarking #6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
OSS-585
The PR is to start using the Galaxy container as the approach to measure performance (only for Meteor staff since we have access to the Galaxy apps involving this repository).
This PR is also a preparation of a proper artillery config that reproduces the regressions found as an action to address community’s feedback on this thread.
If we have this configuration done, other members of the community can also use their infrastructures to deploy the apps and use the script and configuration to compare Meteor 3 and 2 and provide feedback and performance improvements.
The inital approach was using a local-first approach with a fixed machine. It had 64GB of RAM (software capped this memory at its limit), and we noticed the issue reported: Meteor 3 performs worse than Meteor 2, with methods running faster and subscriptions being the slowest. Testing with a remote machine confirmed this behavior, and later we gathered more metrics using APM here. Shifting to a remote-first approach would better align results with the Meteor team and community.