Stickersss
** (RuntimeError) ThousandIsland on HTTP load test
I tested an HTTP endpoint that gives back json like {"user_token":null}.
But, the problem is some requests take too long (which results in 7989 timeouts…), and many warning messages in the logger.
What could be the problem?
-Production- 23:47:02.996 [error] Task #PID<0.12802.0> started from #PID<0.12800.0> terminating
** (RuntimeError) Unexpected error in accept: :emfile
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:42: ThousandIsland.Acceptor.accept/5
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.12800.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:47:02.996 [error] Task #PID<0.13939.0> started from #PID<0.13937.0> terminating
** (RuntimeError) Unexpected error in accept: :emfile
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:42: ThousandIsland.Acceptor.accept/5
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.13937.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:47:02.997 [error] Task #PID<0.13678.0> started from #PID<0.13676.0> terminating
** (RuntimeError) Unexpected error in accept: :emfile
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:42: ThousandIsland.Acceptor.accept/5
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.13676.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:47:02.999 [error] Task #PID<0.13336.0> started from #PID<0.13334.0> terminating
** (RuntimeError) Unexpected error in accept: :emfile
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:42: ThousandIsland.Acceptor.accept/5
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.13334.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
** (stop) exited in: GenServer.call(nil, :acceptor_info, 5000)
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
(elixir 1.16.2) lib/gen_server.ex:1103: GenServer.call/3
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:18: ThousandIsland.Acceptor.run/1
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.67281.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:48:40.104 [error] Task #PID<0.68437.0> started from #PID<0.66288.0> terminating
** (stop) exited in: GenServer.call(nil, :acceptor_info, 5000)
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
(elixir 1.16.2) lib/gen_server.ex:1103: GenServer.call/3
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:18: ThousandIsland.Acceptor.run/1
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.66288.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:48:40.108 [error] Task #PID<0.68560.0> started from #PID<0.66330.0> terminating
** (stop) exited in: GenServer.call(nil, :acceptor_info, 5000)
** (EXIT) no process: the process is not alive or there's no process currently associated with the given name, possibly because its application isn't started
(elixir 1.16.2) lib/gen_server.ex:1103: GenServer.call/3
(thousand_island 1.3.5) lib/thousand_island/acceptor.ex:18: ThousandIsland.Acceptor.run/1
(elixir 1.16.2) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: &ThousandIsland.Acceptor.run/1
Args: [{#PID<0.12353.0>, #PID<0.66330.0>, %ThousandIsland.ServerConfig{port: 4000, transport_module: ThousandIsland.Transports.TCP, transport_options: [ip: {0, 0, 0, 0, 0, 0, 0, 0}], handler_module: Bandit.DelegatingHandler, handler_options: %{opts: %{http_1: [max_header_count: 50], http_2: [max_header_count: 50], websocket: []}, plug: {ChattingWeb.Endpoint, []}, handler_module: Bandit.InitialHandler, http_1_enabled: true, http_2_enabled: true, websocket_enabled: true}, genserver_options: [timeout: 60000], supervisor_options: [], num_acceptors: 1000, num_connections: 1048576, max_connections_retry_count: 5, max_connections_retry_wait: 1000, read_timeout: 60000, shutdown_timeout: 15000, silent_terminate_on_error: false}}]
-Production- 23:48:40.108 [error] Task #PID<0.68740.0> started from #PID<0.65202.0> terminating
** (stop) exited in: GenServer.call(nil, :acceptor_info, 5000)
Load results:
$ wrk -t20 -c9900 -d30s "http://localhost:4000/token" --latency
Running 30s test @ http://localhost:4000/token
20 threads and 9900 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 104.61ms 121.59ms 2.00s 97.84%
Req/Sec 3.68k 676.65 14.18k 79.04%
Latency Distribution
50% 93.00ms
75% 109.48ms
90% 121.73ms
99% 742.60ms
2194409 requests in 30.10s, 0.89GB read
Socket errors: connect 0, read 13492, write 0, timeout 7989
Requests/sec: 72906.82
Transfer/sec: 30.18MB
Marked As Solved
al2o3cr
EMFILE from accept means the process is trying to use more socket connections than it’s allowed to have filehandles.
I’m not 100% certain how that translates into user-facing timeouts, but it certainly isn’t helping the system perform well.
You’ll either need to adjust the limit when starting the BEAM, or adjust the settings of ThousandIsland to not accept as many connections.
1
Popular in Questions
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
Hello all!
I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Hi all,
Trying to get some more clarity over utc_datetime and naive_datetime for Ecto:
https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
I have an umbrella app.
Some of the apps inside depend on other apps in the umbrella, unsurprisingly.
I'm writing a test for one of t...
New
Hey,
I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
I have a User schema with a :from_id field set to type :string:
defmodule TweetBot.Repo.Migrations.CreateUsers do
use Ecto.Migration
...
New
Hello, I have map which I want to convert it to string like this:
the map:
%{last_name: "tavakkoli", name: "shahryar"}
the string I ne...
New
After calling mix ecto.create I get this error:
17:00:32.162 [error] GenServer #PID<0.412.0> terminating
** (Postgrex.Error) FATAL...
New
In AR this is so simple
@articles = current_user.articles
How to do in Ecto?
def index(conn, _params) do
current_user = conn.assig...
New
Other popular topics
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set?
Thanks.
New
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
New
Hi guys, i’m new in the Elixir world, and i have to say, that i love it!
i’m having some problem to understand anonymous functions with ...
New
by Lance Halvorsen
Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
New
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
The deployment platform built for Elixir: PaaS ease, VPS control.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"