Synchronous Messaging at Mozilla - Trial Servers & Feedback

Mozillians
#42

To avoid burying the lede, I’ll come right out and say Riot/Matrix is my first choice and it wasn’t close.

Disclaimer: I’ll admit I didn’t give Rocketchat and Slack the same fair shake as the other two. The former due to poor first impressions and lack of activity. The latter due to previous familiarity and bias (against).

Interface

In terms of the interface, I’d say Riot and Mattermost are a wash. At first I liked Mattermost’s interface better, but after spending a bit of time tweaking my settings I managed to make Riot look much more appealing and less cluttered. I learned that a lot of these settings can have a server default, so we can fix this for people if we want. But overall they both seem to tick most of the major boxes in terms of usability, and getting into the minor details here isn’t very interesting to me.

Moderation

The main reason to switch away from IRC is to make it possible to fight abuse. And here Riot/Matrix seems head and shoulders above the others. One feature I particularly liked is the ability to mute users. This means if anyone is experiencing abuse either in channel or in direct messages, they can take matters into their own hands without needing to wait for an admin/moderator to step in. This feature didn’t seem to exist in Mattermost. Matrix also has a handy moderation guide. While I haven’t tested out everything myself, it looks like Matrix supports many ways to deal with spam and abuse. In contrast I searched the Mattermost docs for “ban”, “abuse”, “kick”, “moderation”, “moderator” but none of those came up with any relevant hits.

Support

All that being said, the biggest draw to me is how dedicated Matthew and the Matrix team are to making this work. From standing up the moderation guide, to fixing bugs in near realtime to simply hanging out and answering questions. It’s clear they are dedicated and willing to support Mozilla (at least for the time being). It has been a very pleasant surprise and I think both orgs will benefit from the relationship (Mozilla gets support, Riot/Matrix gets a large user study). The interactions I’ve witnessed so far have made me confident that any problems Mozilla might have with Riot/Matrix will be taken seriously and addressed relatively quickly.

Extras

I have several other reasons for preferring Riot/Matrix too, though they aren’t criteria in the doc so I’ll just list them quickly.

  • Read-only guest mode
  • E2E Encryption (*not quite fully implemented in Riot yet)
  • Open protocol / not a silo
  • Federation. I know this was a non-goal, but being able to jump into the channels of other open source projects that we interact with (and letting them jump into ours) is useful. If we can enable this without making the moderation story harder, it would be nice to do.
  • IRC / Slack bridges (I think Mattermost supports these too)
  • Other bridges (not so relevant to Mozilla, but nice for individuals who might want them)
  • Several clients to choose from
  • Markdown (Mattermost also has this)

But even without all these extras, from my experimentation Riot/Matrix seems like the best choice for Mozilla.

10 Likes
#43

I agree with everything @ahal said. Riot is also for me my first choice and Mattermost my second.

Here’s one thing I’d add that hasn’t been mentioned but I think is important for me.

App client

An advantage with Riot is that they have a native app available for Android and iOS. The other chat platforms only offer a wrapper of their web version. I’ve tested the Mattermost one and it doesn’t offer a great user experience. Personally, the new platform should have a mobile app that is seen as a first-class citizen by their developers

Also, on desktop there’s a few native clients for Matrix that integrate well with the OS they’re targeting (e.g Fractal with Gnome on Linux). It also has a plugin for Weechat if you like terminal apps.

There’s also Matterhorn for Mattermost but the developers have no intention to integrate SAML/SSO connection and that’s a dealbreaker for me.

6 Likes
#44

Agree with everything said by @nukeador and @ahal about Matrix.

Matrix aligns with Mozilla Mission compared to other options we have, it is a open protocol that enables interoperability and federation, which is a must needed thing to improve the internet health in the current times.

8 Likes
#45

Hi, everyone: we’re now less than 24 hours away from the shutdown of our chat system trial instances. Tomorrow morning - October 9th, morning Eastern time, and with little or no further notice - those machines will be shut down and permanently deleted . As much as I would love to leave these things spinning until it was time to announce the successful candidate, we’ve made a specific commitment about how user data would be handled on these services and we are going to follow through on that commitment.

We’re grateful for everyone’s participation and the feedback we’ve received so far, and we’ve learned a lot. If you have any information from those instances that you would like to preserve, you need to act on that immediately.

Again, thank you for all of your time, effort and feedback. I’m looking forward to the next steps in the process.

6 Likes
#46

And there it is. As announced, we’ve shut down our trial servers.

I’d like to thank everyone who’s participated in this process and given us your feedback. I’m grateful for the time and effort everyone’s put in to testing these services, and we’ve learned a lot.

I hope that we’ll be to distill it all down, make a decision and announce our next steps shortly.

Again, thank you.

4 Likes
#47

I missed the trial (my bad). FWIW there’s some commentary in Reddit:

#48

Slack

I rarely need/needed to use instances of Slack, but the experiences made me uneasy in ways that were difficult to quantify/describe. There was, I think, an underlying sense of “Yet Another System”; plus an overarching sense of things occurring in relative isolation.

Matrix

IHMO this will be a fine choice for Mozilla.

To counter the many plus points, my only significant criticism is of the notifications UX in the https://riot.im/app/ web client.

(For me, the long-standing shortfalls require use of a last-century technology – e-mail – to plug the gaps in an otherwise very pleasantly forward-looking solution. YMMV; I’m in https://riot.im/app/#/room/#riot-web:matrix.org for feedback.)

HTH

Re: Publicly visible streams · Issue #4817 · zulip/zulip and linked issues, I imagine that the current lack of true public visibility was an additional factor in exclusion of Zulip from the final contest.

Also: re: https://chat.zulip.org/#narrow/stream/7-test-here/topic/push.20notifications I couldn’t get push notifications. @rht join me there, if you like; I may be missing something.

#49

The blog post says that “assuming we can gather that support” the new service will be set up on December 1st. Did that happen?

#50

Being Canadian I didn’t account for American Thanksgiving in my calculations. So I won’t have an announcement for you today, unfortunately. I expect (though cannot promise) to have more to say very shortly.

#51

I definitely look forward to us finishing up the transition process. The current schism we have between IRC and Slack has made a mess of communication, with far too many contributor queries being missed because everyone was looking at the other service.

1 Like
#52

Well, you’ll still need to look at two services, since that whatever new thing will not replace the internal Slack. But the interface might be a bit nicer. :wink:

#53

That’s
true – however, currently we have a number of channels set up on the internal slack that shouldn’t be, because people are using it instead of
public channels that had already existed. Hopefully we will strongly encourage those to be closed down in favor of the new public system. Otherwise, our community engagement will continue to suffer as it has the past few months as more people have spend less time looking at IRC and more time looking at Slack.

Indeed, as I’ve watched things the last few months, I’ve seen a number of IRC channels kind of collapse
because the staff that used to be active there have been more focused on Slack instead. This makes me nervous about any ongoing setup in which
we have split chat areas. The more things people have to look at, the more likely they are to either ignore some of them entirely or to at least poorly distribute their time across them.

Since having to start watching Slack in addition to IRC and various GitHub repositories and Bugzilla and the documentation tracking system we have scattered across Trello, Google Sheets, and GitHub, and so on, a lot of stuff has just gotten casually glanced at if at all for months now. There’s just too much stuff in too many places now. We can’t keep adding more things to look at if we expect people to keep up with all of it.

If IRC isn’t good enough for our needs anymore, we need to choose a replacement
that meets our needs well enough that we can minimize or eliminate our Slack usage. The number of user or contributor questions I see going unanswered these days even though there are people around is much higher
than it used to be; I’ve seen it repeatedly today, and it makes me sad.
This didn’t used to happen nearly this much before we had so much reliance on a second chat solution.

We need to be aware of these risks when rolling out a new system.

5 Likes
#54

Since there still seems to be an IRC community: Matrix and Slack can also be bridged to IRC. Don’t know about Slack, but for Matrix that works fine. Matrix users will have a modern UI, while IRC users will receive a link where they can view pictures someone posted on Matrix side.

#55

There is also a very fine Slack bridge for Matrix, that gives you all the nice pretty things like threading, reactions, edits etc :slight_smile:. Also DMs, if you are into that.

(Disclaimer: I maintain it)

2 Likes