Reducing un-targeted Sentry notifications: Fallback changes

[Dhrumil-Sentry]

Sentry informs developers about issues in their app. We have two mechanisms to do this- Issue Alerts and Workflow Alerts.

The email or slack message you receive for a new issue in your Sentry project is sent by Issue Alerts. These notifications are useful to the developer workflow, so Sentry creates an Issue Alert rule by default when a new project is created.

If you look at your alert rules and see a rule called Send a notification for new issues it’s likely that this is a default Issue Alert rule created by Sentry.

This alert rule is meant to inform users for every new issue seen. The action in this alert rule is Send a notification to Issue Owners. So what happens when Sentry cannot identify an Issue Owner? We notify all project members. This is our fallback today. The only way to turn this off is to go to the Project Settings→Issue Owners and turn it off (Yeah, we made it pretty tough to do this !)

There are two problems today:

1. When we don’t find an Issue Owner in a project with more than 10 members, we end up notifying too widely - everyone gets notified and this can get annoying!
2. Finding the fallback setting isn’t easy - it’s not intuitive to have the fallback setting for an Issue Alert action in a Sentry project’s setting.

How do we plan to improve this?

Firstly, we will move the fallback setting to a more logical place - in the Issue Alert rule itself. If you want to send notifications to Issue Owners, you can also let Sentry know who should be notified in case we don’t find an Issue Owner

In addition to All project members and No one, we will support another fallback option:

• Recently Active Members

This will be the default fallback option for Sentry’s default Issue Alert rules

Recently active members are those members of a Sentry project that most recently logged into Sentry - we won’t use any specific look-back period for this.

When the fallback selection for an Issue Alert rule is Recently Active Members, Sentry will notify 20 members at most - These 20 members will include recently active members of the Sentry project in which the issue was observed. We’re choosing to go with 20 as the default based on our analysis of the median and average count of members in a project across Sentry orgs. We believe this default setting will reduce un-targeted notifications from Sentry projects that have a large number of members.

What happens to the Issue Alert rules Sentry already created?

Any Issue Alert rule which has the action Send a notification to Issue Owners will have this action updated to be Send notification to Issue Owners and if none can be found send a notification to Recently active members to reflect the changes we just described.

What if you don’t want anyone to be notified as a fallback and already disabled it in your project settings?

We’ve got you covered. If you’ve already disabled the fallback setting then will change your Issue Alert rule action to Send notification to Issue Owners and if none can be found send a notification to No one accordingly to reflect your selection.

When will these changes go live?

We’re planning to make these changes available for early adopters this month and make this generally available in Jan 2023.

What’s next

We’d love to hear your feedback about anything else we can do to increase the signal-to-noise ratio in our notifications.

Stay tuned for more changes in the coming months that will improve our workflows and give you more value from our notifications.