We’d like to make some decisions around Pontoon based on data in the future.
Our browser support matrix is an obvious candidate. But we’d also like to if our technical developments improve things or not.
In order to see how data can help us here, we first looked at what data we actually currently collect. Note that the Mozilla websites data policy also applies to Pontoon. We have
- server logs
- New Relic for server-side performance metrics
- Google Analytics (GA) for client-side metrics
- Raygun for error Reports
We looked into GA in more details in Munich, and leading up to it. I was in particular interested to find out what our browser support matrix should look like. Data in GA is declining in volume, which is likely due to Tracking Protection blocking the GA script. We don’t see declining usage in the server-side metrics. That makes it a bit hard to reason, and also doesn’t make the numbers we see there useful to share widely. Thus I won’t add them here.
Most of our users are coming to Pontoon with Firefox and Chrome. In the timeframe that I looked at, there was a pattern of new users in Chrome, and then new users in Firefox, and then staying 
The long tail is Edge, Safari, IE 11. And some of you use SeaMonkey.
As for our browser support matrix, I want Pontoon to use something that I call watersheds. We’ll declare the list of browsers we support, and the minimum version as an explicit version. If we want to use a browser feature that needs a newer version, we’ll look at the data at that point, and decide on whether it’s OK to move the requirements up or not.
One big question is going to be mobile browsers. Right now, our mobile support on Pontoon is pretty bad, so usage is low. But when we develop new experiences with mobile in mind, we hope to change that. So we’ll need to anticipate our browser support story for mobile.
I know that there’s libraries out there that key off of “last two versions” or so. I’ve spent some time digging through the dependency hell of those libraries, and they end in data sources I have no confidence in. Thus, we’ll make decisions based on our own data, and we’ll do every now and then.
Axel