Great idea, I’d love to see some benchmarks of all of them.

We have some slightly out-of-date ones for some of the above here.

image600×371 21.5 KB

image600×371 20.9 KB

Mist came out fastest of the BEAM servers, and could sometimes beat Go depending on request body size.

One thing that really surprised me was how badly Cowboy handled request bodies. JavaScript was beating it as soon as there was a reasonable amount of data to read!

I’d expect to see similar numbers in a benchmark for Bandit and Elli since they both use Erlang’s http parser – which also means Bandit will have the same scaling issue Cowboy hit years ago that resulted in cowlib being created.

But for many many use cases Elli (or Bandit I suppose) work great

Bandit author here. Late to the party, but a couple of things:

1. I’ve looked for historical evidence of decode_packet’s supposed scalability issues but I can’t seem to find anything. Can you point me at some references?

2. Performance numbers in (synthetic) benchmarks suggest quite the opposite regarding the scalability of cowlib vs. decode_packet. When I was building out GitHub - mtrudel/network_benchmark I had to give up on higher concurrency tests as I was unable to get Cowboy to complete them without massive error counts, whereas Bandit hummed along just fine (and indeed grew the performance gap even more on higher concurrency tests).

m.

The only references I can find specifically to decode_packet being replaced by cowlib dates back to 2013: https://groups.google.com/g/erlang-programming/c/C9OYrllYkpI/m/DeKOVx3rqEAJ

Some perf assertions also from that era: https://groups.google.com/g/erlang-programming/c/tJnDTcgb9L8/m/zd2fpAnCIyYJ

Considering the vintage of those assertions (R16-era), I don’t think they’re terribly relevant any longer, at least not without being corroborated by more contemporary comparisons. This proverbial ‘best before’ date is even more apparent given that benchmarks done both by me and Rawhat (GitHub - rawhat/http-benchmarks: benchmarks for mist, and other webservers) indicate that Cowboy’s performance hasn’t kept up.

The 0.6 release train of Bandit is going to be focused entirely on observability, performance and reproducible benchmarking. I expect to get an additional 20% or so perf gains out of Bandit based on this work, which should make it the fastest game in OTP town by a pretty significant measure. Expect work to start late 2022.

m.

There was a Plug but I don’t think it was ever updated for the last major release of Plug.

My understanding is it won’t work as a Plug anymore because it works different from Cowboy. I vaguely recall looking once, not too deeply, and think it was technically possible but would be a hack.

It also is “fast” because it isn’t completely general purpose – it only supports bodies up to a staticly defined size and requires a plugin to include the datetime in the response, things like that.

Other reason it is fast is it uses the builtin Erlang http parser (C code). But Cowboy replaced this with cowlib for a reason, scaling. cowlib will scale up on cores and keep better latency as concurrent requests go up.