fidzu : thunderbird

We recently released the telemetry alerting beta, and announced it in the blog post here! This blog post will dive into the details of how it works across Treeherder, and Mozdetect. At a high level, MozDetect handles the change point detection for telemetry probes, and Treeherder handles storing the detections, and producing the emails/bugs for these.

MozDetect

All of the existing, and any future change detection point techniques used for telemetry alerting are built in MozDetect. Having these live outside of Treeherder gives a low-barrier to entry for adding new features, and testing existing ones without having to set up everything needed for alerting in Treeherder. It's built as a python module that is run through uv. This makes it very easy for anyone to run the code because of uv's excellent python version, and dependency management. How to work with the code in this repository is outlined here, along with how to add your own techniques to it (note the access to mozdata through gcloud is required for this).

Detectors are split into two parts: (i) a detector that performs a comparison between two groups, and (ii) a detector that performs detection on a time series (using the detector from (i)). Our default detection technique, called cdf_squared lives here. The timeseries_detector_name is the name that will be used to access the detector from the telemetry probe side through the change_detection_technique field. The only method that absolutely needs to be implemented by these is the detect_changes method and it must return a list of Detection objects. These detection objects contain all the necessary information for producing an alert. There is also an optional_detection_info field that can contain additional things like attachments that would be added to Bugzilla bugs, and additional_data that can hold JSON data for storage in the DB. The cumulative distribution function (CDF) squared technique uses these to store the CDF before and after the detection along with a graph of these as an attachment for the Bugzilla bug.

CDF Squared Detection Technique

The CDF squared technique detects changes in time-series histogram data by comparing CDFs between consecutive windows. It takes two CDFs, each representing the distribution of measurements over a time window, and computes the sum of squared differences between the two CDFs at each bin. The sign of the summed linear difference is then used to assign a direction to the squared difference score so that the output encodes whether the distribution moved to higher values (right shift) or lower values (left shift).

For time-series detection, this base comparison is applied in a rolling fashion across the full history of data. Each day's 7-day smoothed CDF is compared against the next one, producing a continuous signal of squared CDF differences over time. A Butterworth low-pass filter is then applied to that signal to remove high-frequency noise while preserving genuine trend changes. Finally, scipy's find_peaks function is used to locate statistically significant peaks and valleys in the filtered signal using a dynamic alert threshold based on the historical data. Information is extracted from those areas and then used to build the detection information needed for the alert generation process.

Alerting

Our alerting tooling lives in the Treeherder codebase. It's run through our PerfSheriff Bot (called Sherlock) and runs once per day. When a detection is produced from MozDetect, a telemetry alert is added to the database and then the TelemetryAlertManager is called to handle it. The manager's tasks are split into 6 ordered phases:

1. Update alerts with changes from Bugzilla. This step ensures that any changes that happen in the bugs filed are mirrored into our database. Currently, we only track resolution changes here.
2. Comment on existing bugs. This step is for updating existing bugs with information from new alerts. This step is not currently being used. In the future, this could be used to inform probe owners that a probe which doesn't produce bugs has produced an alert in the same time range.
3. File new bugs for alerts. This step handles filing bugs for any new alerts on probes set up for producing bugs.
4. Modify existing bugs with new alerts. This step handles any modifications needed to existing bugs based on the new bugs that were created. Currently, the "See Also" field is modified for existing bugs to include the new bugs.
5. Produce emails for new alerts. This step handles producing emails for any alerts set up to produce emails.
6. Housekeeping. This step handles redoing any failures that happen above in either the current run or past runs. Currently, it's being used to retry bug modifications and sending emails when we encounter a failure there. This excludes retrying bug filling since we delete the alert in that case and retry it the next time the alert is generated.

After the housekeeping step, the manager is done for the day and runs again on the next day to handle any updates and new alerts. Contrary to how alerting works for performance tests in CI, this process is fully automated and requires no human input at any point.

Setting up telemetry probes for alerting happens on the mozilla-central side in their probe schema using the new monitor field in the metadata section (example for email alerts, example for bug alerts). The telemetry alerting documentation has information about how to do this. We then use an index.json file from the telemetry dictionary to gather all the probes that should be alerting. The information there is supplemented by more granular information later in the pipeline to gather things like the time unit used for the probe to be able to better format the Bugzilla bug table.

Once a telemetry probe is set up for alerting and is found by our system, the owners (those listed in the email notification fields) will begin either receiving emails or have bugs produced for them. These can also be viewed by everyone on this dashboard.

Acknowledgements

Getting the project to this point involved work from people across multiple teams here at Mozilla. Special thanks to Eduardo Filho for his support on the telemetry probe side, to Bas Schouten for his guidance and work on

the CDF Squared detection technique, and to Andrej Glavic and Beatrice Acasandrei for their help in reviewing the Treeherder-related changes.

If you hit any issues with the telemetry alerting system, or have any suggestions feel free to file a bug in the Testing :: Performance component or reach out to us in either #perf-help on Slack or in #perftest on Matrix.

22 Apr 2026 12:40am GMT

We're happy to announce that the Telemetry Alerting beta is now open to everyone!

Monitoring for changes in telemetry probes that you own can be difficult to do on a regular and continuous basis. With telemetry alerting, that changes today! You can now quickly set up your timing distribution probes for automated monitoring on Windows with notifications through email or a Bugzilla bug.

To get started, if you only need email alerts, simply add monitor: True to the metadata section of your probe (example).

If you would prefer to receive Bugzilla bugs when a change is detected, set the monitor field like so (example):

monitor:
alert: True
lower_is_better: True/False # Optional
bugzilla_notification_emails:
- <YOUR-BUGZILLA-EMAIL-HERE>

More information about telemetry alerting, and how to set up a probe can be found here in the documentation. There's also a dashboard that can show you all of the existing telemetry alerts along with some detection information. For now, we only support change detection on Windows for `timing_distribution` probes (see here for other desktop platforms, and android).

Please note that this is an open beta and we are actively looking for feedback on this system. If you hit any issues, or have any suggestions feel free to file a bug in the Testing :: Performance component or reach out to us in either #perf-help on Slack or in #perftest on Matrix.

Special thanks to Eduardo Filho for his support on the telemetry probe side, to Bas Schouten for his guidance and work on the CDF Squared detection technique, and to Andrej Glavic and Beatrice Acasandrei for their help in reviewing the Treeherder changes.

For a more detailed look at how this works, see this blog post.

21 Apr 2026 7:58pm GMT

Mobile browsing hasn't kept up with how people actually use their phones.

Right now, even basic tasks can feel harder than they should. Finding what you need can mean scrolling through ads and filler content, keeping track of too many tabs, or thinking twice about how private your connection is.

A mobile browser should do more - and we're raising the bar. Firefox is rolling out a set of updates that build on our most popular desktop features and adapt them for how you browse on-the-go. Here's what's out now, and what's coming next.

Get the key points with Shake to Summarize

When you're following a recipe, reading a product review, or deciding whether a long article is worth your time, getting to the useful part can take longer than it should.

With Shake to Summarize, you can shake or tap your phone to generate a quick summary of the page. Currently available for iOS users in English, we're expanding availability to all iOS users in German, French, Spanish, Portuguese and Italian starting with Firefox 150 on April 21. We'll also soon be making Shake to Summarize available to Android users in English, so they too can get to the key points of any article in seconds.

Take control of how AI shows up

AI features are becoming a more common part of browsers - but not everyone wants the same experience. Firefox gives you a say in how they're used. With AI Controls, you can turn AI features off entirely, enable only the ones you want, or adjust things over time. Rolling out on Android and iOS beginning May 21.

Stay protected with a free, built-in VPN

Firefox's free built-in VPN covers up to 50 gigabytes of your browsing in Firefox each month, across desktop and mobile devices. It adds a layer of protection to your browsing activity by masking your IP address - especially useful when you're on public Wi-Fi. Unlike many "free VPNs" that rely on ads or selling user data to generate revenue, Firefox is built with a different model: no selling your browsing data, no injecting ads into your traffic. Instead, we offer a limited amount of browser-level protection for free, alongside Mozilla VPN, our paid, unlimited, full-device VPN service. Rolling out on Android soon.

Keep your tabs organized with Tab Groups

Tab Groups have been among the most-requested mobile features from our Mozilla community, and they're coming on mobile soon. You'll be able to group related tabs to stay organized, whether you're comparing restaurants, planning a trip or saving articles to read later.

We're also building toward smart groupings, where Firefox can automatically suggest tab groups for you. Rolling out on Android soon.

More updates, built around how you browse on mobile

Your phone comes with a browser. That doesn't mean it has to stay your default

"Firefox exists to give people a better way to experience the web, and that has to be just as true on mobile as it is on desktop," said Ajit Varma, head of Firefox. "For many people, their phone is their primary way of getting online, and they deserve a browser that's fast, intuitive and built around their needs. That's why we're investing in mobile more than ever before. We're building for the millions of people who choose Firefox every day, and giving even more people a reason to do the same."

Firefox is building a mobile experience designed around how people browse - with tools that help you move faster, stay organized and stay in control.

These updates begin rolling out in April with more on the way.

The post What's new in Firefox mobile: Less clutter, more control and a free built-in VPN appeared first on The Mozilla Blog.

21 Apr 2026 7:36pm GMT