There's no good IM - Open IM is failing

Flamboyant article with flamboyant title. Here we go: All current open IM (oIM) solutions offered as alternative to Discord are bad.
This article is going to be rather brutal because the time to do a fluff article about this is over. Right now, there is a significant OSS gap for replacing Discord. Discord has established itself as the de-facto IM platform for years now, Discord is now looking to squeeze their users for money, we need a way out of this abusive relationship.

I will create some lingo here just so we're on the same page.
I consider there to be 3 modes of operation for chatting:

• 1:1 - This is direct messages, you talk with one person. This is usually one of the easiest to design.
• 1:some - This is 'group chats' or essentially a IRC channel. Here more than 2 people come together to chat into a singular room, this can often scale up to hundreds of participants. Protocols / E2EE systems made with 1:1 in mind usually start failing here.
• 1:many - This is the Guilds / ""Server"" paradigm from Discord. Compared to the previous category, while we have possibly thousands of people in one space, 1:many spaces contain multiple channels that are logically sorted and moderated under 1 entity (i.e. they are not independent separate channels like you might see for IRC channels).

Discord trumps because it mastered the first two and popularized the last. OSS has not caught up and has many many problems along the way.
I'm sorry for the negativity in this article, this is a whine article. There are plenty of things these solutions do right, this is listing my gripes with them rather than praising them.

Pick your poison #

Matrix #

When you bring up Discord alternatives, Matrix will almost always come up and it does check a lot of boxes in terms of Discord features, but my time with Matrix has been really bad. I can't really recommend it in good faith, even if I wish I could.

Potential conflict of interest with Element? #

In 2023, Element overtook the development of Synapse (current going homeserver implementation) and Dendrite (Golang based homeserver implementation) from the Matrix foundation. This seems like a conflict of interest to me, as Element is a for-profit entity unlike the Matrix foundation.
Although, Element really plainly states in their own announcement of the takeover:

As founders of Matrix, we created Element as a for-profit open source company to hire the core Matrix team to be able to work on Matrix, develop a flagship Matrix-based product, bootstrap the Matrix ecosystem, and help fund the underlying core Matrix projects. As a commercial entity, Element has driven the bulk (more than 95%) of core Matrix development for the last seven years, and maintains the largest Matrix homeserver (matrix.org) on behalf of the Matrix.org Foundation.

This is not an immediate issue. In Element's AGPL announcement, they also quite plainly state:

Now, CLAs come in all shapes and sizes: some good and some bad. It’s critical to understand that our reason for requiring one here is to give us the right to sell AGPL exceptions: not to “do a Hashicorp” and switch to a non-FOSS licence in future.

At least from the current perspective of things, they don't appear to be in it to get a large userbase and then squeeze it.

But I'd be lying if the thought in the back of my head wasn't there:
Element currently controls and maintains the main server implementation for Matrix, implementations such as Dendrite and Conduit (independent Rust implementation) exist but they're plagued with interoperability issues such as Dendrite failing signature checks against Synapse servers. Clearly developing your own Matrix server is a herculean task which even Element doesn't manage to get right.

How would the Matrix ecosystem respond to hostilities from Element? Aren't we putting our eggs into one basket?

This is something which will never get an answer until it actually happens.

Matrix 2.0 #

In September of 2023, the Matrix foundation has unveiled the idea of Matrix 2.0. My phrasing was very careful there because Matrix 2.0, to the acknowledgement of the foundation itself at the tail end of 2023, Matrix 2.0 is not actually a formalized protocol version yet but a set of independent improvements.

One of the more important changes is Sliding Sync, a proposal which introduces a way for clients to filter and sort the rooms they are joined to and then request a subset of the resulting list of rooms rather than the entire room list.
This is required because Matrix's current sync method does not scale well to larger amounts of rooms, causing clients to spend a long time on syncing.

Noble, technically difficult goal. The idea for Sliding Sync has been around for as long as 2021 I believe and nowadays is implemented using the Sliding Sync Proxy.

I will be blunt: As far as I'm aware, there is 1 client supporting this: Element X, Element's flagship beta client.

(That's technically incorrect, there is SchildiChat Next but it simply forks Element X, so I consider it from the same heritage)

I'm sure this will improve as it gets more stable and established over time, but for now the entire thing seems to be an idea that entirely exists in the ballpark of Element. I bring this up independently of the centralization concern, even if they feel similar.

Clients, damned Clients, and Implementation Sprawl #

On Nheko and Element, I can view threads, although in entirely different paradigms with Element giving me a more Discord-style thread selection, where each Thread acts more as its own segregated channel (that's good!). While on Nheko, it's more like a reply chain with a UI wrapper around it, no way to enumerate the Threads as far as I know.
On Cinny, this is just not a thing. Not implemented yet.

This section is weak and not actionable by nature of being extensible, you can't (or rather shouldn't) push easily non-backwards incompatible changes and you need work on actually implementing the features. But it's not enjoyable to me as a user and I've seen a good amount of complains along the lines of "Why can't I do emoji?".

Proposal process is painful #

I've not been around Matrix for long but the time spent waiting for the proposal process to do its thing is equally as painful as it must be for older users.

There are a lot of features missing that make the experience of using it frustrating:

• Clients notify you when you're obviously at your devices
• Matrix doesn't allow you to customize your profile, you only get a username and an avatar.
  • This has been a topic for almost 10 years since writing, with an issue being migrated from Jira in 2015
  • The closest solution appears to be MSC1769: Extensible profiles as rooms from 2019
    • Which at least externally appears stagnant? I'm sure there's technical reasons for it, but they're not really accessibly laid out without reading through the entire thread
    • There was a bump on the status in 2021 but this seemingly got no reply
    • The way that Matrix manages proposals is also painful here, the code review threads end up feeling very cluttered when trying to browse the thread
      • This is not really their fault though per-se, I think this is still fairly accessible and this still feels rather homey as developer.
  • Mind you, XMPP had figured out profile customization back in 1999....
• Matrix has a form of custom emotes and stickers as mentioned before but they're not formalized yet
  • Matthew Hodgson states Element missing emojis and stickers is due to the competing proposals for it, making Element bide their time instead at the frustration of their users
  • MSC2545 has become the going implementation for this over time and is becoming slowly on track to get formalized
• There is no way to self-destruct a server in order to tell other servers that they cannot federate with this server anymore.
  • Mind you, I'm not expecting a 100% clean exit, that's not possible because of federation.
  • But having some sort of decommission protocol would be nice.

Some proposals are more minor are fine being left on the table for a bit, but features such as profile customization are a critical in my opinion. I've been misgendered from time to time by accident on Matrix because I don't have the ability to show my identity to others. I can change my name to include pronouns for example, but I consider that more of a workaround.
This entire thing wouldn't be an issue if I could attach it to my profile, link to a place for this on it or explain them in a biography. I never had this issue on Discord because I have the ability to customize my profile and that allows anybody who cares enough to introspect on me by, for example, reading my website.

And that's ignoring absolute mess that the Pronouns proposal was, I digress.

A lot of these issues and proposals have been sitting around for a good while, externally appearing stagnant.
All of this really doesn't feel good, although I think that's easy to say with the naivete of not actually being involved in the process.

This ends up being a chicken and egg problem. Proposals are stable when they get merged into the spec, at which points their implementation becomes absolute, but until then proposals are in a state of uncertainty which leads developers (including Element) to only reluctantly work with them.

The best suggestion I can provide is that MSCs could represent a formal blocking status (the blocking label does exist but it doesn't tell you much) or relationship between MSCs, so that users can see at a glance what issues have to be implemented for this MSC to be implementable and track the status of it as a whole. A lot of these MSCs are frustrating because at a glance, they appear to be sitting idle.

XMPP has a thing called compliance suites which lists out categories of proposals to implement. This gives developers a target and users something to compare feature sets. This concept doesn't particularly work well because more or less this happens when a proposal becomes part of the Matrix spec.
Matrix.org kind of helps by listing out what features clients support, but clearly pain points remain.

Soatok's complaints about cryptography #

I'm not qualified enough to talk about cryptography, and you may validly dismiss this point for appealing to authority.
I would be dishonest if I would not bring up the writeup from Soatok Dreamseeker, who knows a lot more about cryptography than I do.

While they mostly only reiterate a research paper that found the vulnerabilities in the first place, I think the entire thing probably degrades my trust in the cryptography for Matrix until a new security audit happens. Read up on it and decide for yourself.

Discord-Matrix bridging is hell #

Oh boy, I get to put on a hat. I develop AdGoBye which is an Adblocker or Content blocker for Social VR.
We've needed to create a social space for development purposes. We like the idea of Matrix because it's more open than Discord would be for this, we're a FOSS project and we prefer FOSS for our infrastructure obviously. The original plan was to maintain both Discord and Matrix spaces and bridge them together, this is a mode of operation that's encouraged by the foundation itself but this ended up being a bad experience to the point that it made me erase this section which was supposed to commend Matrix for bridging.

t2bot.io #

As recommended in the page linked, we started out with t2bot.io which runs matrix-appservice-discord. This was fairly nice because they provide the infrastructure for us, we simply link the bot to our spaces and we're done....? You'd wish, we probably came at a bad time.

The bot would intermittently stop bridging which was a huge deal to us since it made holding a conversation flaky when your conversation partner could go away without a notice.
Message edits are done pretty done in a ugly manner:

Occasionally duplicating messages

Or sending the message with the edited content again with a link preview, interrupting the flow of conversation.
A member of the server pointed out that webhooks, which is the mechanism of how "puppeting" works on Discord's side, does support editing messages finally which means you can edit messages.

You can't unbridge your space short of administrator intervention, requiring contacting the admins at t2bot.io or whoever hosts the bot to help you if you end up misconfiguring that.

According to this issue, you can't bridge Discord's announcement channels, which is a feature that allows Discord servers to push announcements to other servers.

Those are some of the largest problems that the bridge has right now, I'm sure we'll find more the longer we use it.

Is anybody home? #

The reason why it's like this is because upstream appears to have been hit by a bus.

As of writing, the last commit was 8ae61dd which was 10 months ago.
t2bot.io actually maintains their own fork tracking that, which is ahead by 17 commits but also had its last commit 9 months ago.

Scrolling through the issues and matrix space of the Appservice, there are plenty of users who require triage on issues but are answered by nobody.

The saddest part some of the issues listed actually have PRs to be fixed:

• 916 - chore: remove embed on discord edits
• 819 - feat: M -> D native discord edits
• 735 - Unbridge portals
• 889 - Add Discord Announcement channels to allowed bridging types

But there's nobody to review and merge them, no acknowledgement on the state of things officially on the situation that I was able to find.
Someone will have to fork the appservice, not me because I unfortunately don't have the time or patience for it.

The rest of the story #

Anyway, matrix.org lists three other bridges.
One is mautrix-discord, which I actually use which puppets an entire Discord user, so not what we need.
Another is matrix-discord-bridge, which is unmaintained.

I ended up trying Out of your Element which contains more features and was made with efficiency in mind.
While the bridge worked mostly fine over its time, it's intended to be used for a canonical Discord space, which would be managed there but allows a no-strings-attached fully managed bridging over to Matrix. That's awkward, because we already have a Matrix space.

I hosted it on my own homeserver, so as administrator, I hijacked the bot's channels to be owned by me and added it to our real Matrix space.

Mission accomplished? Would have been, but at some point the bot started to not bridge messages anymore. I think it might have been because of this, I'm not sure. To be honest due to the thing mentioned above, I don't really want to continue using this either.

Which leads me to today, where I'm considering abandoning support for the Discord server in favor of Matrix, probably to the detriment to my users.

Hell is paved with good intentions #

I think there is potential but the overall picture is bitter enough for me to not buy into it. Realistically knowing my friend circles though, I'll be forced to stick around Matrix by Network effect (the irony is palpable).

If you're willing to take a one-armed protocol and accept that change will be very slow, possibly being fine with the current status quo, then Matrix might be the solution for you. But I cannot sign it off in the current state.

XMPP #

I'll be upfront about my biases, XMPP is pretty much my choice if Discord explodes tomorrow. I like the protocol and it has been battle hardened over the years, like an IRC with more features, but I'd be lying if it didn't hurt.

No 1:many - probably from IRC #

XMPP unfortunately due the time it was developed is stuck with the 1:some model. You get Multi User Channels (MUCs), which give you a singular channel to talk into which multiple people get. Your server ends up being the host (i.e IRC network) for that channels and you enumerate channels on a per-server basis.

Similar to IRC, you can sort of emulate 1:many by splitting your channels into different channels, but they're not visually grouped under one space and instead end up exposing 5 different channels as separate entities. The problem with this that it creates channels that just exist for the meta-organization, which are treated as essentially the same worth as the real channels.

They don't really feel cohesive (because they were never meant to be that way), which is why 1:many is so important.

I lack the technical phrasing for this but I also feel like 1:some doesn't scale well in XMPP? Messages take a while to sync, lots of requests happen if you're in a busier MUC like Gajim's chat. Maybe this is closer to a UI/UX problem than a technical one, but I think it's better if protocols were designed with 1:many in mind rather than as an afterthought. Feel free to dismiss this point if I'm just wrong.

The 2000s wants its clients back #

XMPP being an older protocol ends up inheriting that in the UI/UX client language as well.
A lot of it feels like its emulating the design language of things like Whatsapp, which makes sense for the audience they were going for at that time but I personally feel like doesn't hold up too well nowadays.
Notable examples include Dino and Kaidan.

Converse.js breaks the mold a bit by imitating KiwiIRC instead, which might be a bit backwards but honestly doesn't look too bad:

I think the only place where I prefer this paradigm is on Mobile, although even Conversations has since then implemented a more modern Material design:

Talking about it with a friend, I personally feel like this creates a monoculture for clients, because I would find it difficult to not recommend Gajim (screenshots) and Conversations (see repo for screenshots) to newcomers as they both count as some of the most mature XMPP clients in the ecosystem.
I feel like I would be setting up someone for failure if I didn't onboard them with those, and that's not a good sign.

Frozen in time #

XMPP feels like its stuck back in an older time.
Some features I believe XMPP needs to compete:

• 1:many
  • Provide a way to categorize channels into a singular entity instead of having them be separate
  • Provide moderation and permission management based on this singular entity
  • Ideally design-wise separate 1:many from 1:some and 1:1
• Custom emojis (small user submitted inline images)
  • Sounds silly but they add a lot in terms of expression, most people coming from Discord will affirm this to you.
  • Provide a way for the user to upload images to the server (or do something similar to Fedi and have your administrator manage them instead) with a shortcode to reference it
  • Then you can type :ying_science: in chats and it turns it into (cute yinglet source)
  • Clients not supporting this should just see the :shortcode: in place instead
  • These images aren't expected to change too much so in order to lighten the load on servers, clients should probably be encouraged to aggressively cache this, maybe impose a file size limit.
• Link previews
  • This is a serious creature comfort, having a way to introspect what the link they're heading to is really useful
  • How to implement this has already been figured over the years
  • Based on this converse.js issue, I believe the mechanism for this would be XEP-0367 ?
  • I think that this is missing is on people's radars:
    • I stumbled onto Dino's Google Summer of Code project for this
    • Gajim appears to have had a plugin for this, seems removed from the repo nowadays
• Expanded styling
  • XEP-0393 specifies styling for italics, bold text, Strikethrough and Monospace (that's good!).
  • Discord but also Matrix support a greater syntax in comparison:
    • Spoilers, where there is a specification in XEP-0382 (though currently deferred)
    • Bullet point lists (can you tell I like them)
    • "Ordered" i.e. numbered lists
    • Headings (I started out thinking they're dumb for IM but they help with structure for longer messages, similarly to the web content)
• Extended presence?
  • Probably leaning too hard into Discord-oIM interop but it would be nice to show more in-depth info about what I'm doing
  • XMPP only shows me my current availability and a short text about it
    • Like come on, even ICQ had more granularity: (sarcasm in case it wasn't obvious)
      
  • A thing I like about Discord is that I'm able to quickly see at a glance if my friends are up to, like what they're coding or how the game they're playing is going
  • It allows me to see if they might be busy or if I can join in for some activity
  • The feature was made for gaming in mind, because that's for who Discord started out as, but since then mostly molded into a generic form of 'rich' i.e. extended Presence
  • None else supports something like this, XMPP can be ahead of the curve in this regard
  • This has way more technical depth required to implement, so I struggle to give much implementation suggestions aside from pointing at Discord's Rich Presence
  • Update: XMPP folks have informed me about XEP-0107: User Mood, XEP-0108: User Activity, XEP-0118: User Tune and XEP-0196: User Gaming which are all extensions to provide enhanced status.
    • The fact that I missed them probably shows that I'm not around for too long :^)
    • This obviously erodes my point and this makes me in retrospect look a bit silly, I'll keep it here with this note.
    • Maybe these extensions could be genericized in some form? It'd be interesting to see why there is lack of interest for it.

Nitpick: Dead mailing lists? #

Update: Daniel Gultsch themselves has chimed in to explain that most conversation happens over XMPP (hi I'm flattered), hence the lack of activity on the mailing lists. Thanks!

XMPP.org has a list of mailing lists, however when you actually check the Hyperkitty GUI for them, only the ones involved in the standardization process are actively used. This is pretty confusing to me and I don't really understand why it's this way, are there actually just no sysadmins talking about XMPP administration for example?

Trailblazer left in the dust #

It's been a good while since XMPP has been big in the 2000s but it feels like it never recovered since, I will still prefer it out of a probably incorrect sense of 'technical correctness' compared to Matrix. Although I'm not sure if I'm not just becoming those IRC holdouts, just for XMPP.

Signal #

I personally think that Signal doesn't qualify for this discussion. It competes with the likes of Telegram and Whatsapp, I am only including it so that I won't get techbro'd about it.
I see a lot of evangelization of Signal which annoys me a lot. The people who evangelize Signal around where this post is shared get their car hit with exploding hammers. Please read the room.

It's fine. The cryptography is world class, the Server and Client are open sourced and have a good amount of eyes on it. The UI/UX is solid and takes the usual Whatsapp style UI approach, it works but it's also not really what we're looking for. I don't like that it requires a phone number, I don't like that their server has zero documentation on selfhosting, I don't like that the client exposes as far as I know no way to switch out the server (even for us techies) if you were to host your own server.

I think that centralizing everything on their servers will end up biting them for growth, especially as a non-profit that's reliant on donations. I think this might shatter the evangelization of Signal in the long run, Telegram had way too much money to subsidize them for user growth and even they're starting to squeeze their users for money. Signal will have to beg their users for donation, find alternative revenue streams for their service (such as creating a for-profit entity similar to Mozilla) or die trying.

IRC #

To be honest, I don't want to write about it.
Read this article by James Lu instead because it actually contains the complaints of someone who more actively uses IRC than I do. This sums up my thoughts on it with more authority. There's plenty I could add but I don't feel confident in writing about it.

IRC as pioneer also does not do 1:many, see my thoughts about this paradigm at the XMPP section.

Misery #

Here's the call of action: The time to develop better oIM is now. Discord has been getting more hostile for a while and is beginning with pestering their users, the tensions will continue until they inevitably mess up and cause a user exodus.

That's the time we should be using to draw people onto open alternatives instead of another centralized platform like Guilded or Revolt, instead oIM continues to be on the backfoot catching up with the design philosophies that Discord established a decade ago.

I think what I am asking for is essentially impossible, I want a OSS instant messaging solution with the tight UI/UX of Discord but that isn't as centrally tied to a company like Element is. Doing that is way way way easier said than done, I'd call it essentially impossible. I don't have a solution for this, this is not an article where I shill my own project so we become adversaries against all the other solutions mentioned.

But I can pretty plainly state that existing solutions feel bad, because of diverse technical and social issues. I will still use them and I commend everyone who has the patience to be flexible because I highly doubt there will be a one glove fits all solution.

But please, whatever happens, please let's not repeat Discord.
Discord became big because Microsoft started burning their good faith with the Skype userbase,
so we switched to Discord and now Discord is burning their good faith with us.

How often should we repeat this until we realize where it leads?