Release Health (Session Tracking)

[smeubank]

Hey Sentry JS world!

What do you think about our Release Health (session tracking) product for Javascript applications?

We want to improve our JS release health data, to make the release health feature more useful for JS developers.

So let us hear it - the good, the bad, and the ugly! 😄

1. Is the release health feature useful to you?
2. If not then why?
3. Are we collecting the right kind of data? Do we communicate the health of your app in a way that is meaningful?
4. What additional metrics might you want to see, to track the health of a Javascript release?
5. Are the answers to the above different between Node and browser, or in different frameworks?

Examples:

• If we stick with the idea of crash-free sessions, what should count as a “crash” (especially for web apps, considering that browsers don’t provide a good API for capturing when a tab hard-crashes)? There is no clear fix, but we'd like to hear your thoughts.

• Right now we mark error events as “handled” or “unhandled” (with tags and in the UI of the issue stream). Do you pay attention to this, and find the distinction helpful? If you do, what are your use cases?

Maybe you haven't seen or tried out Release Health yet? Now is a great time.

https://sentry.io/resources/release-health-by-sentry/

https://docs.sentry.io/platforms/javascript/configuration/releases/

[danielasy]

Hello! I'm currently looking into front-end health metrics, so I'm very interested in Release Health.

I'm a fairly recent user though. We have a few small apps using it, and I'm going to test it against larger projects with more events and issues in the next weeks.

However, some questions popped up while I was reading the docs and exploring the dashboard. Some of them have to do with how we already track errors, some of them with how crashes and "handled" vs "unhandled" work. Please bear with me if I've missed something in the way :)

• Errors caught with a React Error Boundary: many of our apps have one application-level error boundary, or they are built with Next, which has a global error component. When an error is caught by an error boundary, the entire component tree below is unmounted, and the error is manually captured. Therefore, it's treated as handled and doesn't count as a crash. However, this scenario is not the most ideal UX, as everything the user was seeing is replaced with an error page. Currently, we are setting these errors to level:fatal severity, so we can query these issues separately and treat them with priority, but we lack visibility in how much they're affecting sessions and users.
• Errors in event handlers: as far as I understand, @sentry/browser has an automatic instrumentation to enhance event data of errors happening in event handlers. This means these errors are all handled, despite the application code doing nothing to deal with them. I would like to measure their impact in some way, as "button doesn't work" can be quite critical in some cases.
• Errors in third-party API integrations: another use case I've seen is capturing errors due to 4xx and 5xx responses from third-party APIs. When it's not possible to set up alarms directly in the vendor platform, it might be easier to spot problems such as exceeded quota limits using Sentry events and alarms. This one is kind of an odd situation, but I'll leave it here just in case.
• Sessions with error: the docs say that the session is marked as "an error if the SDK captures an event that contains an exception (this includes manually captured errors)". Is "Error Free Session" something we are able to see? I think that would be pretty helpful and could go along nicely with crash metrics.

I'll share more impressions once I get Release Health going in more projects. In the meanwhile, I'd welcome any insight about these points!

[smeubank]

getsentry/sentry-replay#38

for reference

[vladanpaunovic]

Today, JS session tracking is not ideal and it seems like we are trying to reinvent the wheel here.

What we consider a “session” and what a regular user would consider a “session” are two distinct topics with different definitions.

The most popular SDK that has session tracking out there is Google Analytics and they have a very clear definition of what a session is, when it starts, how long it lasts and where it ends.

Given the user volume of GA, I believe that users would perceive the session as whatever Google defines it to be. On Javascript, we could try aligning with them to provide more value.

Why align with Google Analytics' definition of a session?
As a website owner who has both Sentry and Google analytics injected
I want to have a consistent definition of sessions across platforms
So I get a more accurate understanding of the traffic volume of my releases

Recommended read: How a web session is defined in Universal Analytics