agacode

agacode

Performance issues (slowness) after elixir upgrade

Hi Guys,

I’m have a performance issue after an elixir upgrade:

before:
elixir 1.6.4
erlang 20.3
phoenix 1.3.2

now:
elixir 1.9.0
erlang 22.0.4
phoenix 1.4.10

2000 requests/sec, 100 DB Connections (Postgres)

Before the 2000 requests executed in about 4 - 5 min, but the average time for each request was like 6s (I know this number is high and I need to cache some results to lower the request execution time, but that is not the problem I’m facing)

After the upgrade the total time is less about 2 - 3 min, but the average time for each request is 40s. It was a surprise to me that the new Elixir with the same settings as before was performing worse, even more, I’m losing some requests the HAProxy (working as reverse proxy) shows 2000 and my elixir backend app shows 1600. I even had to tweak some configuration values (to be able to finish most of the request) that I didn’t have to tweak before.

When I look at the logs locally, I see that the DBConnection Pool is switching too much between request and it is trying to execute close to 1 query on each request. So, it means that I have close to 2000 request executing little by little (I think because of this new CoDel alg). I would prefer to use a setting (If there was one) to assign a DBConnection to a request till the request finishes executing. I was looking also into DBConnection.Ownership to play with the :ownership_timeout but I also read that DBConnection.Ownership is more for tests (might have misinterpreted this)

My current settings:

	config :myapp, BeaconWeb.Endpoint,
	  http: [
		...
		protocol_options: [
		  idle_timeout: 1_200_000,
		  inactivity_timeout: 1_200_000
		]
	  ]

	config :myapp, Beacon.Repo,
      ...
	  pool_size: 100,
	  timeout: 100_000_000,
	  queue_target: 100_000_000

As additional information I’m getting this message frequently on localhost:
With DBConnection.ConnectionPool:

[info] [] Postgrex.Protocol (#PID<0.2384.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.4135.0> exited

With DBConnection.Ownership:

[error] [] Postgrex.Protocol (#PID<0.2384.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.4135.0> exited

Documentation I have checked:
https://hexdocs.pm/db_connection/DBConnection.html
https://hexdocs.pm/db_connection/DBConnection.Ownership.html
https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html
https://ninenines.eu/docs/en/cowboy/2.5/manual/cowboy_http/
https://ninenines.eu/docs/en/ranch/1.6/manual/ranch/

Any help will be appreciated. Thanks in advance.

Most Liked

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

@agacode Usually if you wanted to do that “app wide” you’d basically just do it at the controller or plug call function or whatever. Doing it app wide is a tricky concept with all the different processes running around. Most processes never even touch the database, so there has to be an explicit point at which you say “This process is checking out a database connection”.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

Regular. If your responses are taking 40 seconds though I’d seriously audit the database requests being made. Could you also supply the actual mechanism you’re using to benchmark?

l00ker

l00ker

This suggestion might be going out on a limb… but have a look at this recent post:

Disable prepared statements for an individual query

I didn’t bother to research when prepared statements where introduced in ecto etc., but since this was such a large upgrade jump, maybe you’re being affected in a similar way and it’s adding up fast due to the shear volume of queries.

It’s a long shot, but the read and eventual outcome might prove educational either way :slight_smile:

dimitarvp

dimitarvp

@agacode :point_up: This. Give us some more details, there are practically zero at this point.

Where Next?

Popular in Questions Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs &amp; Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement