Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add telemetry #4441

Open
wants to merge 15 commits into
base: develop
Choose a base branch
from
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ input/*

# simulation outputs
out/
config.py
matplotlibrc
*.pickle
*.sav
Expand Down
3 changes: 3 additions & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,6 @@ file:///home/runner/work/PyBaMM/PyBaMM/docs/source/user_guide/fundamentals/pybam

# Errors in docs/source/user_guide/index.md
file:///home/runner/work/PyBaMM/PyBaMM/docs/source/user_guide/api_docs

# Telemetry
https://us.i.posthog.com
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
# [Unreleased](https://github.com/pybamm-team/PyBaMM/)

## Features

- Added basic telemetry to record which functions are being run. See [Telemetry section in the User Guide](https://docs.pybamm.org/en/latest/source/user_guide/index.html#telemetry) for more information. ([#4441](https://github.com/pybamm-team/PyBaMM/pull/4441))
- Added sensitivity calculation support for `pybamm.Simulation` and `pybamm.Experiment` ([#4415](https://github.com/pybamm-team/PyBaMM/pull/4415))

## Optimizations

- Removed the `start_step_offset` setting and disabled minimum `dt` warnings for drive cycles with the (`IDAKLUSolver`). ([#4416](https://github.com/pybamm-team/PyBaMM/pull/4416))

## Features

- Added phase-dependent particle options to LAM #4369

## Breaking changes

- The parameters "... electrode OCP entropic change [V.K-1]" and "... electrode volume change" are now expected to be functions of stoichiometry only instead of functions of both stoichiometry and maximum concentration ([#4427](https://github.com/pybamm-team/PyBaMM/pull/4427))
- Renamed `set_events` function to `add_events_from` to better reflect its purpose. ([#4421](https://github.com/pybamm-team/PyBaMM/pull/4421))

Expand Down
9 changes: 9 additions & 0 deletions docs/source/user_guide/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,12 @@ glob:
../examples/notebooks/creating_models/5-half-cell-model.ipynb
../examples/notebooks/creating_models/6-a-simple-SEI-model.ipynb
```

# Telemetry

PyBaMM collects anonymous usage data to help improve the library. This telemetry is enabled by default but can be easily disabled. Here's what you need to know:

- **What is collected**: Basic usage information like PyBaMM version, Python version, and which functions are run.
- **Why**: To understand how PyBaMM is used and prioritize development efforts.
- **Opt-out**: To disable telemetry, set the environment variable `PYBAMM_OPTOUT_TELEMETRY=true` or use `pybamm.telemetry.disable()` in your code.
- **Privacy**: No personal information (name, email, etc) or sensitive information (parameter values, simulation results, etc) is ever collected.
1 change: 1 addition & 0 deletions examples/scripts/compare_lithium_ion.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import pybamm

pybamm.set_logging_level("INFO")
pybamm.telemetry.disable()
valentinsulzer marked this conversation as resolved.
Show resolved Hide resolved

# load models
models = [
Expand Down
1 change: 1 addition & 0 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ def set_iree_state():
"IREE_INDEX_URL": os.getenv(
"IREE_INDEX_URL", "https://iree.dev/pip-release-links.html"
),
"PYBAMM_OPTOUT_TELEMETRY": "true", # don't capture telemetry in tests
}
VENV_DIR = Path("./venv").resolve()

Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ dependencies = [
"typing-extensions>=4.10.0",
"pandas>=1.5.0",
"pooch>=1.8.1",
"posthog",
]

[project.urls]
Expand Down Expand Up @@ -280,3 +281,6 @@ module = [
"pybamm.models.full_battery_models.base_battery_model.*"
]
disable_error_code = "attr-defined"

[tool.poetry.scripts]
post-install = "scripts.post_install:run"
valentinsulzer marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 5 additions & 0 deletions scripts/post_install.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import pybamm


def run():
pybamm.config.generate()
6 changes: 4 additions & 2 deletions src/pybamm/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@
# Batch Study
from .batch_study import BatchStudy

# Callbacks
from . import callbacks
# Callbacks, telemetry, config
from . import callbacks, telemetry, config

# Pybamm Data manager using pooch
from .pybamm_data import DataLoader
Expand All @@ -205,6 +205,7 @@
"batch_study",
"callbacks",
"citations",
"config",
"discretisations",
"doc_utils",
"experiment",
Expand All @@ -220,6 +221,7 @@
"simulation",
"solvers",
"spatial_methods",
"telemetry",
"type_definitions",
"util",
"version",
Expand Down
85 changes: 85 additions & 0 deletions src/pybamm/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
import uuid
import pathlib
import os


def _get_config_file():
# Get the home directory
home_dir = pathlib.Path.home()

Check warning on line 8 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L8

Added line #L8 was not covered by tests

# Create the file path for pybamm config
config_file = home_dir / ".config" / "pybamm" / "config.yml"

Check warning on line 11 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L11

Added line #L11 was not covered by tests
valentinsulzer marked this conversation as resolved.
Show resolved Hide resolved

return config_file

Check warning on line 13 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L13

Added line #L13 was not covered by tests


def is_running_tests():
"""
Detect if the code is being run as part of a test suite.

Returns:
bool: True if running tests, False otherwise.
"""
import sys

# Check if pytest or unittest is running
if any(
test_module in sys.modules for test_module in ["pytest", "unittest", "nose"]
):
return True

# Check for GitHub Actions environment variable
if "GITHUB_ACTIONS" in os.environ:
return True

Check warning on line 33 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L32-L33

Added lines #L32 - L33 were not covered by tests

# Check for other common CI environment variables
ci_env_vars = ["CI", "TRAVIS", "CIRCLECI", "JENKINS_URL", "GITLAB_CI"]
if any(var in os.environ for var in ci_env_vars):
return True

Check warning on line 38 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L36-L38

Added lines #L36 - L38 were not covered by tests

# Check for common test runner names in command-line arguments
test_runners = ["pytest", "unittest", "nose", "trial", "nox", "tox"]
return any(runner in sys.argv[0].lower() for runner in test_runners)

Check warning on line 42 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L41-L42

Added lines #L41 - L42 were not covered by tests


def generate():
if is_running_tests():
return

Check warning on line 47 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L46-L47

Added lines #L46 - L47 were not covered by tests

# Check if the config file already exists
if read() is not None:
return

Check warning on line 51 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L50-L51

Added lines #L50 - L51 were not covered by tests

config_file = _get_config_file()

Check warning on line 53 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L53

Added line #L53 was not covered by tests

# Create the directory if it doesn't exist
config_file.parent.mkdir(parents=True, exist_ok=True)

Check warning on line 56 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L56

Added line #L56 was not covered by tests

# Generate a UUID
unique_id = uuid.uuid4()

Check warning on line 59 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L59

Added line #L59 was not covered by tests

# Write the UUID to the config file in YAML format
with open(config_file, "w") as f:
f.write("pybamm:\n")
f.write(f" uuid: {unique_id}\n")

Check warning on line 64 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L62-L64

Added lines #L62 - L64 were not covered by tests


def read():
config_file = _get_config_file()

Check warning on line 68 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L68

Added line #L68 was not covered by tests

# Check if the config file exists
if not config_file.exists():
return None

Check warning on line 72 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L71-L72

Added lines #L71 - L72 were not covered by tests

# Read the UUID from the config file
with open(config_file) as f:
content = f.read().strip()

Check warning on line 76 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L75-L76

Added lines #L75 - L76 were not covered by tests

# Extract the UUID using YAML parsing
try:
import yaml

Check warning on line 80 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L79-L80

Added lines #L79 - L80 were not covered by tests

config = yaml.safe_load(content)
return config["pybamm"]
except (yaml.YAMLError, ValueError):
return None

Check warning on line 85 in src/pybamm/config.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/config.py#L82-L85

Added lines #L82 - L85 were not covered by tests
3 changes: 3 additions & 0 deletions src/pybamm/simulation.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import warnings
from functools import lru_cache
from datetime import timedelta
import pybamm.telemetry
from pybamm.util import import_optional_dependency

from pybamm.expression_tree.operations.serialise import Serialise
Expand Down Expand Up @@ -450,6 +451,8 @@ def solve(
Additional key-word arguments passed to `solver.solve`.
See :meth:`pybamm.BaseSolver.solve`.
"""
pybamm.telemetry.capture("simulation-solved")

# Setup
if solver is None:
solver = self._solver
Expand Down
41 changes: 41 additions & 0 deletions src/pybamm/telemetry.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
from posthog import Posthog
import os
import pybamm
import sys

_posthog = Posthog(
# this is the public, write only API key, so it's ok to include it here
project_api_key="phc_bLZKBW03XjgiRhbWnPsnKPr0iw0z03fA6ZZYjxgW7ej",
host="https://us.i.posthog.com",
)


def disable():
_posthog.disabled = True


_opt_out = os.getenv("PYBAMM_OPTOUT_TELEMETRY", "false").lower()
if _opt_out != "false":
disable()


def capture(event):
# don't capture events in automated testing
if pybamm.config.is_running_tests():
return

properties = {

Check warning on line 27 in src/pybamm/telemetry.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/telemetry.py#L27

Added line #L27 was not covered by tests
"python_version": sys.version,
"pybamm_version": pybamm.__version__,
}

config = pybamm.config.read()
if config:
user_id = config["uuid"]

Check warning on line 34 in src/pybamm/telemetry.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/telemetry.py#L32-L34

Added lines #L32 - L34 were not covered by tests
else:
user_id = "anonymous-user-id"
properties["$process_person_profile"] = False

Check warning on line 37 in src/pybamm/telemetry.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/telemetry.py#L36-L37

Added lines #L36 - L37 were not covered by tests

# setting $process_person_profile to False mean that we only track what events are
# being run and don't capture anything about the user
_posthog.capture(user_id, event, properties=properties)

Check warning on line 41 in src/pybamm/telemetry.py

View check run for this annotation

Codecov / codecov/patch

src/pybamm/telemetry.py#L41

Added line #L41 was not covered by tests
Loading