halfmetaljacket

halfmetaljacket

Optimizing WebSocket Compression in Plug Cowboy: Reducing CPU Overhead on High-Traffic Socket Server

I’m facing a peculiar challenge with a socket server I’ve built using Elixir and Cowboy WebSocket (Cowboy WebSocket documentation). This server has been in production for a while, handling substantial traffic. It consumes messages from RabbitMQ, processes them, and publishes messages to clients based on their subscribed channels.

The issue arises with data-out costs. To tackle this, I enabled built-in compression in Cowboy. However, the problem is that messages are compressed separately for each client. For instance, if a message needs to be sent to 1000 clients, it gets compressed 1000 times, one for each client process. This approach has caused high CPU overhead and spiking latencies, especially during message bursts.

To address this, I’m considering an alternative:
Pre-compressing messages when they’re consumed from RabbitMQ and sending the pre-compressed messages directly to clients that support compression. For clients that don’t support compression, the original uncompressed message would be sent instead. The plan is to add relevant headers so that clients (mostly web browsers) can automatically decompress messages without requiring any changes on the frontend.

However, I’m unclear about how this approach interacts with WebSocket compression features like server_context_takeover, server_max_window_bits, etc. Since Cowboy optimizes compression by managing compression contexts across frames, how would this work when the messages are already pre-compressed?

Has anyone encountered a similar problem or implemented a solution for this? I assume this is a common challenge for socket servers serving public data.

Any insights, best practices, or ideas to optimize CPU and latency in this scenario would be greatly appreciated!

Edit: GoLang’s Gorrila Websocket has a functionality called PreparedMessage that will solve my issues. But plugging this functionality into the cowboy library is way beyond my skill. I can try to implement it when I have some free time.

Most Liked

rjk

rjk

Given your problem I think you’re on the right track to pre-compress the payload you want to send. I would not use the deflate options of the websocket itself as this is always a 1:1 relation. I would assume there’s a single point in your app where you broadcast from the single rabbitmq message to multiple (websocket) receivers. That would be the place in your code to do the one-time compression and send it out to all the receivers as a binary frame.

I’ve made a lot of websocket implementations for high traffic websocket servers and a lot of them implemented compression of their payload but none of them used the built-in compression extension of websockets (rfc7692), they just send it out as binary frame and specify in their documentation which form of compression they have used. Another reason I would not opt for the built-in compression is that a client can always ignore it or express that it doesnt support compression (following the standard negotiation flow) and opt for the uncompressed stream and that would undo what you want.

Another thing which can also help you a lot with burst traffic is to implement some form of buffering and sending data out in “ticks”. In my implementation I sometimes had to handle 1000s of messages per second and then I would just buffer everything within 250ms of 1sec and compress and send it out once per that time interval. I was not able to give back pressure to my (external) upstream so this helped a lot with high traffic spikes and not fill up the mailboxes of the involved actors/processes.

HTH

(EDIT1: I just see that there’s way more context about your issue on reddit, link to that post: https://www.reddit.com/r/elixir/comments/1hn91p6/need_help_in_optimizing_websocket_compression/ )

EDIT2: super interesting post about what the compression levels do in reality: Configuring & Optimizing WebSocket Compression - igvita.com

VictorGaiva

VictorGaiva

Your issue seems extremely related to what Discord went through, and blogged about a few months ago. Take a look How Discord Reduced Websocket Traffic by 40%

mpope

mpope

This does not answer the question of how to serve shared compressed values, but have you tried changing the deflate options to adjust compression settings?

A more experimental option would be to create your own zlib context and emulate what Cowboy does but return the raw frame {binary, iodata()} from the websocket handler directly, where the payload is pulled by sequence number from a shared ETS cache.

halfmetaljacket

halfmetaljacket

I did a POC for this for, the compressed payload almost doubled after increasing just one level, which is fine with me. Yet to do a proper benchmark.

Where Next?

Popular in Questions Top

ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New

Other popular topics Top

JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New

We're in Beta

About us Mission Statement