All,

After several years of effort, I just published version 1.0.0 of both the Bandit and Thousand Island libraries. Folks that are depending on versions in the 0.x.y or 1.0.0-pre series of either library should update your dependencies to be ~> 1.0.

This has been a ton of work, and has been made possible in large part due to the help of tons of contributors. In particular, @moogle19, @ryanwinchester and @alisinabh have gone above and beyond on all fronts. The project wouldn’t be the success it is without help from folks like them. Thanks all!

I put together a bit of a retrospective blog post about the whole journey here, if anyone cares to learn more!

Excellent question and one I get a lot. The short answer is that in most cases you probably won’t see much of a difference between Bandit and Cowboy from a performance perspective; your plug’s implementation is going to be the dominant factor in overall performance, and switching out the underlying server won’t magically make that work go away.

That having been said, there are many workloads in which you could expect to see a benefit to Bandit. The ideal case would be large numbers of HTTP/1 clients doing lots of IO on very short lived connections. In that case you could see some substantial benefits (see my latest benchmark for more).

Some workloads are going to be worse. In particular, HTTP/2 performance in Bandit is pretty awful at the moment, but is going to be getting a lot of attention as part of the work to add WebSockets over HTTP/2 (RFC 8441) support. This will be one of the next things I’m working on.

In terms of LiveView, Bandit’s WebSocket implementation is generally a little bit faster than Cowboy’s (around 10-20%). You might see some real-world benefit there; it really depends on your particular usage patterns.

Tough to tell. A few observations:

• BLUF: Of course this is an apples and oranges comparison. I personally couldn’t give two hoots how well we perform against other languages (especially natively compiled ones running on base libraries); that’s not a game I have any interest in playing or one that has any winners. It’s MONGODB IS WEB SCALE all over again and I have better things to do with my time than to engage in the comparative aspects of this. The two contestants aren’t even playing the same sport

• The PR’s setup looks fine (it’s not a matter of app configuration). I didn’t look at any of the lower level OS / BEAM tuning details

• He’s using m7a.large instances, which at first glance look like they’d perform a smidge worse than the instances we use for microbenchmarks in CI. From that perspective the results seem roughly correspondent with what I’d expect in absolute terms.

• I’m a little worried by the growth numbers that Bandit demonstrates. CPU usage shouldn’t be growing without bound like that, and (as he states) that’s likely the root cause of the lacklustre numbers elsewhere

• There’s not really a whole lot of actionable steps to take based on this data. We really do need a better benchmarking environment (ideally one that runs as part of CI), as the microbenchmarking setup we use now just doesn’t get to the absolute scale needed to reproduce these sorts of situations ‘in the lab’, which is a necessary precondition to be able to improve them in Bandit. If anyone is looking for a place to help, that’s probably the highest value way to do so

I know this is controversial but we are doing real HTTP in the tests

To me, not controversial at all. I use the following very very often:

{:ok, pid} =
  Bandit.start_link(
    plug: fn conn, _ -> Plug.Conn.send_resp(conn, 200, "ok") end,
    port: 0
  )
{:ok, {_ip, port}} = ThousandIsland.listener_info(pid)
url = "http://localhost:#{port}"

This is the 80% Bypass I most commonly need. In fact I’d like to be able to use Req.Test mocks backed by Bandit somehow, if that makes sense, but I couldn’t find the right API. Apologies for hijacking the topic, I’ll bow out now but if it piques anyones interest I’m happy to talk through it further elsewhere.

The behaviour I’m trying to mimic (remote server crashing) is not sending connection: close headers, so i’m not sure it would be a good test here, but I may try.

Ah! That’s an oversight on Bandit’s part (RFC9112§9.6 says we should be sending that header before closing the connection if possible, and it’s easily possible to do so).

Just added this behaviour to Bandit, if you want to test against main!