-
-
Notifications
You must be signed in to change notification settings - Fork 534
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add telemetry #4441
base: develop
Are you sure you want to change the base?
Add telemetry #4441
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #4441 +/- ##
========================================
Coverage 99.41% 99.42%
========================================
Files 293 295 +2
Lines 22554 22599 +45
========================================
+ Hits 22423 22468 +45
Misses 131 131 ☔ View full report in Codecov by Sentry. |
We could also make |
Yes, but we also want to disable it when our dependencies run their tests, so that wouldn't be enough right? |
Oh, yes, forgot about that! In that case, we would need both the logic you added and the fixture to call it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand why we want to know which parts of PyBaMM our users are using, but I am not sure this is the best approach.
My concerns:
- I feel like I would be upset to find out a free research tool was tracking my usage and sending it to a server. I think for a paid tool the expectation that they want to know your usage is a little more reasonable.
- Users installing through pip might not realize we are tracking their usage. It would be better to allow an opt-in instead so that users know we are collecting their data.
- Are we only going to be tracking simulation data so that we can just keep it in the simulation class or are we going to have to add the tracking to a lot of different places and make sure we add it to new functionality?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @valentinsulzer. I suggest a timeout with default opt-out so we don't break ppl's CI, see below
"This is entirely optional and does not impact the functionality of PyBaMM.\n" | ||
"For more information, see https://docs.pybamm.org/en/latest/source/user_guide/index.html#telemetry" | ||
) | ||
while True: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this should time-out after a short period with default no so we don't break everyone's automated stuff (say 10-20 sec?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I agree, we should not have an infinite loop here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A better option could be to check for the ci_env_vars
above (can be set as a constant at the top of the file) and assume non-interactive if set, and return early (like nox
does it with session.interactive
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is only ever called by generate
which already returns early if it detects an automated environment, but I'll also add the time check as a backup
src/pybamm/config.py
Outdated
) | ||
while True: | ||
user_input = input("Do you want to enable telemetry? (Y/n): ").strip().lower() | ||
if user_input in ["yes", "y"]: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if user_input in ["yes", "y"]: | |
if user_input in ["yes", "y", ""]: |
So if user presses enter it will be interpreted as a yes (as per the prompt)
Description
Adds very basic telemetry - records when a simulation is solved.
The following section has been added to the user guide to explain the telemetry we are doing: