KDE Crash Tracking System 💣
KDE is now evaluating Sentry, a crash tracking system.
Who can get access? Everyone with a KDE developer account.
But what is it?
Since forever we have used Bugzilla to manage crash reports but this has numerous challenges that haven’t made any improvements in at least 10 years:
- Finding duplicates crashes is hard and in our case involves a human finding them
- When debug symbols are missing we need to ask the user to recreate the problem, which is not always possible
- Users need to worry about debug symbols (this is in part improved by the rise of debuginfod - yay!)
- We have no easily consumed graphs on how prevalent a specific crash is, and by extension we have a hard time judging the importance
- The user needs to actually write a report for us to learn of the crash (spoiler: most crashes never get this far)
- …
All in all it’s a fairly dissatisfactory situation we are in currently. Enter Sentry.
Sentry is a purpose-built crash tracking system. It receives crash reports via API ingestion points and traces missing frames with the help of debuginfod, can detect duplicates automatically and thus show us particularly aggressive crashes, and much more. Best yet, it supports many different programming languages which allows us to not only improve the quality of our software but also our infrastructure services.
The current evaluation instance is already amazing and helped fix numerous problems, and the current setup is not even using all features yet and we have hampered rollout a bit: only git builds currently submit data. If all goes well and we find it to be amazing I hope we’ll eventually be able to roll this out to production releases.
Let’s look at a crash I’ve fixed recently.
Here’s what Sentry received from the user:
Not terribly useful. So with the power of debuginfod it turned it into this:
I then applied some brain power to create a fix and consequently the crash has disappeared, as we can see in this neat graphic here:
Here’s a complete crash information page from a recent infrastructure problem in our bugzilla bot:
Also check out my Akademy talk: