Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(integrations): add support for cluster clients from redis sdk #2394

Merged
merged 13 commits into from
Dec 7, 2023
20 changes: 14 additions & 6 deletions sentry_sdk/integrations/redis/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
)

if TYPE_CHECKING:
from collections.abc import Callable

Check warning on line 16 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L16

Added line #L16 was not covered by tests
from typing import Any, Dict, Sequence
from redis import Redis, RedisCluster
from redis.asyncio.cluster import (

Check warning on line 19 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L18-L19

Added lines #L18 - L19 were not covered by tests
RedisCluster as AsyncRedisCluster,
ClusterPipeline as AsyncClusterPipeline,
)
Expand Down Expand Up @@ -141,12 +141,15 @@


def _set_db_data(span, redis_instance):
# type: (Span, Redis) -> None
_set_db_data_on_span(span, redis_instance.connection_pool.connection_kwargs)
# type: (Span, Redis[Any]) -> None
try:
_set_db_data_on_span(span, redis_instance.connection_pool.connection_kwargs)
except AttributeError:
pass # connections_kwargs may be missing in some cases

Check warning on line 148 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L147-L148

Added lines #L147 - L148 were not covered by tests


def _set_cluster_db_data(span, redis_cluster_instance):
# type: (Span, RedisCluster) -> None
# type: (Span, RedisCluster[Any]) -> None
default_node = redis_cluster_instance.get_default_node()
if default_node is not None:
_set_db_data_on_span(
Expand All @@ -155,15 +158,20 @@


def _set_async_cluster_db_data(span, async_redis_cluster_instance):
# type: (Span, AsyncRedisCluster) -> None
# type: (Span, AsyncRedisCluster[Any]) -> None
default_node = async_redis_cluster_instance.get_default_node()
if default_node is not None and default_node.connection_kwargs is not None:
_set_db_data_on_span(span, default_node.connection_kwargs)


def _set_async_cluster_pipeline_db_data(span, async_redis_cluster_pipeline_instance):
# type: (Span, AsyncClusterPipeline) -> None
_set_async_cluster_db_data(span, async_redis_cluster_pipeline_instance._client)
# type: (Span, AsyncClusterPipeline[Any]) -> None
_set_async_cluster_db_data(
span,
# the AsyncClusterPipeline has always had a `_client` attr but it is private so potentially problematic and mypy
# does not recognize it - see https://github.com/redis/redis-py/blame/v5.0.0/redis/asyncio/cluster.py#L1386
async_redis_cluster_pipeline_instance._client, # type: ignore[attr-defined]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way we could try to avoid using _client here, and instead access what we need through a public API? I would strongly prefer to avoid using an external library's private API, since the private API could change at any time and cause problems in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't believe there is a way we can avoid using _client. We can use a try/except here to be safe and log when it is unavailable. Is there a standard way to raise sentry internal warnings?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, in that case, we can use capture_internal_exceptions here. I also checked that one of your unit tests is verifying this function, so we will get a test failure if _client ever gets removed

)
szokeasaurusrex marked this conversation as resolved.
Show resolved Hide resolved


def patch_redis_pipeline(pipeline_cls, is_cluster, get_command_args_fn, set_db_data_fn):
Expand Down Expand Up @@ -237,7 +245,7 @@
except AttributeError:
pass
else:
patch_redis_pipeline(

Check warning on line 248 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L248

Added line #L248 was not covered by tests
strict_pipeline, False, _get_redis_command_args, _set_db_data
)

Expand Down Expand Up @@ -310,13 +318,13 @@
except ImportError:
pass
else:
patch_redis_client(

Check warning on line 321 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L321

Added line #L321 was not covered by tests
rb.clients.FanoutClient, is_cluster=False, set_db_data_fn=_set_db_data
)
patch_redis_client(

Check warning on line 324 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L324

Added line #L324 was not covered by tests
rb.clients.MappingClient, is_cluster=False, set_db_data_fn=_set_db_data
)
patch_redis_client(

Check warning on line 327 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L327

Added line #L327 was not covered by tests
rb.clients.RoutingClient, is_cluster=False, set_db_data_fn=_set_db_data
)

Expand All @@ -328,7 +336,7 @@
except ImportError:
return

patch_redis_client(

Check warning on line 339 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L339

Added line #L339 was not covered by tests
rediscluster.RedisCluster, is_cluster=True, set_db_data_fn=_set_db_data
)

Expand All @@ -340,7 +348,7 @@
# https://github.com/Grokzen/redis-py-cluster/blob/master/docs/release-notes.rst
if (0, 2, 0) < version < (2, 0, 0):
pipeline_cls = rediscluster.pipeline.StrictClusterPipeline
patch_redis_client(

Check warning on line 351 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L351

Added line #L351 was not covered by tests
rediscluster.StrictRedisCluster,
is_cluster=True,
set_db_data_fn=_set_db_data,
Expand All @@ -348,7 +356,7 @@
else:
pipeline_cls = rediscluster.pipeline.ClusterPipeline

patch_redis_pipeline(

Check warning on line 359 in sentry_sdk/integrations/redis/__init__.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/__init__.py#L359

Added line #L359 was not covered by tests
pipeline_cls, True, _parse_rediscluster_command, set_db_data_fn=_set_db_data
)

Expand Down
12 changes: 7 additions & 5 deletions sentry_sdk/integrations/redis/asyncio.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,16 @@
from sentry_sdk.utils import capture_internal_exceptions

if TYPE_CHECKING:
from collections.abc import Callable
from typing import Any
from typing import Any, Union
from redis.asyncio.client import Pipeline, StrictRedis
from redis.asyncio.cluster import ClusterPipeline, RedisCluster

Check warning on line 19 in sentry_sdk/integrations/redis/asyncio.py

View check run for this annotation

Codecov / codecov/patch

sentry_sdk/integrations/redis/asyncio.py#L16-L19

Added lines #L16 - L19 were not covered by tests


def patch_redis_async_pipeline(
pipeline_cls, is_cluster, get_command_args_fn, set_db_data_fn
):
# type: (type, bool, Any, Callable[[Span, Any], None]) -> None
# type: (Union[type[Pipeline[Any]], type[ClusterPipeline[Any]]], bool, Any, Callable[[Span, Any], None]) -> None
old_execute = pipeline_cls.execute

async def _sentry_execute(self, *args, **kwargs):
Expand All @@ -45,11 +47,11 @@

return await old_execute(self, *args, **kwargs)

pipeline_cls.execute = _sentry_execute
pipeline_cls.execute = _sentry_execute # type: ignore[method-assign]


def patch_redis_async_client(cls, is_cluster, set_db_data_fn):
# type: (type, bool, Callable[[Span, Any], None]) -> None
# type: (Union[type[StrictRedis[Any]], type[RedisCluster[Any]]], bool, Callable[[Span, Any], None]) -> None
old_execute_command = cls.execute_command

async def _sentry_execute_command(self, name, *args, **kwargs):
Expand All @@ -67,4 +69,4 @@

return await old_execute_command(self, name, *args, **kwargs)

cls.execute_command = _sentry_execute_command
cls.execute_command = _sentry_execute_command # type: ignore[method-assign]
22 changes: 19 additions & 3 deletions tests/integrations/redis/cluster/test_redis_cluster.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ def monkeypatch_rediscluster_class(reset_integrations):
"localhost", 6379
)
pipeline_cls.execute = lambda *_, **__: None
redis.RedisCluster.execute_command = lambda *_, **__: None
redis.RedisCluster.execute_command = lambda *_, **__: []


def test_rediscluster_breadcrumb(sentry_init, capture_events):
Expand All @@ -29,7 +29,15 @@ def test_rediscluster_breadcrumb(sentry_init, capture_events):
capture_message("hi")

(event,) = events
(crumb,) = event["breadcrumbs"]["values"]
crumbs = event["breadcrumbs"]["values"]

# on initializing a RedisCluster, a COMMAND call is made - this is not important for the test
# but must be accounted for
assert len(crumbs) <= 2
if len(crumbs) == 2:
assert event["breadcrumbs"]["values"][0]["message"] == "COMMAND"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mention in the comment, initializing the RedisCluster executes a COMMAND call to the redis server to get available commands. This only happens on the first test run of a parameterized run (more of a problem for test_rediscluster_basic), perhaps there is some caching mechanism somewhere

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rewrite the assertions as follows:

Suggested change
# on initializing a RedisCluster, a COMMAND call is made - this is not important for the test
# but must be accounted for
assert len(crumbs) <= 2
if len(crumbs) == 2:
assert event["breadcrumbs"]["values"][0]["message"] == "COMMAND"
# on initializing a RedisCluster, a COMMAND call is made - this is not important for the test
# but must be accounted for
assert len(crumbs) in (1, 2)
assert len(crumbs) == 1 || crumbs[0]["message"] == "COMMAND"

The first assertion should be changed because the test should explicitly verify that at least one breadcrumb was created; the previous assertion would have succeeded with no breadcrumbs and the test would only fail later with an IndexError.

Regarding the second assertion, I think my suggested revision conveys more clearly the intent to verify that either there is only one breadcrumb, or if there are two, then the first breadcrumb should be a command.

If you prefer the structure with the if statement for the second assertion, it is okay with me to keep that structure, but in either case, please replace event["breadcrumbs"]["values"][0]["message"] with crumbs[0]["message"]


crumb = event["breadcrumbs"]["values"][-1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
crumb = event["breadcrumbs"]["values"][-1]
crumb = crumbs[-1]


assert crumb == {
"category": "redis",
Expand Down Expand Up @@ -65,7 +73,15 @@ def test_rediscluster_basic(sentry_init, capture_events, send_default_pii, descr
rc.set("bar", 1)

(event,) = events
(span,) = event["spans"]
spans = event["spans"]

# on initializing a RedisCluster, a COMMAND call is made - this is not important for the test
# but must be accounted for
assert len(spans) <= 2
if len(spans) == 2:
assert event["spans"][0]["description"] == "COMMAND"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My previous comment applies here, as well


span = event["spans"][-1]
assert span["op"] == "db.redis"
assert span["description"] == description
assert span["data"] == {
Expand Down
Loading