steffend

steffend

Phoenix Core Team

Plug.Conn.read_body vs. Plug.Conn.chunk performance

Hello there,

I’m currently working on implementing a WebDAV server using Plug/Cowboy, inspired by the great Python wsgidav library. So far read only access works fine and when downloading files using Plug.Conn.chunk I’m seeing very good performance (1,88GB/s - GigaBYTES from local SSD to /dev/null) with less than 50% scheduler utilization:


But while implementing initial write support I’ve noticed that the other direction, using Plug.Conn.read_body maxes out at about 200MB/s - even when just discarding the output:

You can try this yourselves by cloning the repository (write_support branch) and running the demo project from the demo folder. Then starting the server is as easy as using iex -S mix after the usual mix deps.get step.
For uploading, you can use curl -X PUT -F 'data=@/dev/random' http://demo:demo@127.0.0.1:5001/tmp/test.null.
For downloading, first create/download a sufficiently large file and then use wget -O /dev/null "http://127.0.0.1:5001/tmp/10G" --user demo --password demo (replace /tmp/10G with the file path).

I tried tweaking the Plug.Conn.read_body opts, but did not find anything that improved this.

Maybe someone with deeper insight into plug/cowboy has an idea why the Plug.Conn.read_body direction performs so bad. I’d expect both directions to have roughly the same performance and scheduler usage.

Thanks for any hints!

Most Liked

steffend

steffend

Phoenix Core Team

That’s not the case. As documented, read_body reads the body up to a given length. So both are processing the data in chunks. Otherwise the memory usage would explode when reading indefinitely from /dev/random.

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement