cjk
Ecto timeout errors when wrapping into cachex calls in tests
Hi there,
I use Cachex to cache very frequent queries:
@spec get_setting_for_forum(%Forum{}) :: %Setting{} | nil
def get_setting_for_forum(%Forum{} = forum) do
Cachex.fetch!(:cforum, "settings/forums/#{forum.forum_id}", fn ->
from(setting in Setting, where: setting.forum_id == ^forum.forum_id)
|> Repo.one()
end)
end
@spec get_setting_for_user(%User{}) :: %Setting{} | nil
def get_setting_for_user(%User{} = user) do
Cachex.fetch!(:cforum, "settings/users/#{user.user_id}", fn ->
from(setting in Setting, where: setting.user_id == ^user.user_id)
|> Repo.one()
end)
end
This function loads a settings object from the database. It works fine in production and dev code, but in tests I get this error:
** (Cachex.ExecutionError) connection not available and request was dropped from queue after 602ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
I call the getter functions like this:
def load_relevant_settings(forum, user)
def load_relevant_settings(nil, nil) do
[get_global_setting()]
|> Enum.reject(&is_nil(&1))
end
def load_relevant_settings(%Forum{} = forum, nil) do
[get_global_setting(), get_setting_for_forum(forum)]
|> Enum.reject(&is_nil(&1))
end
def load_relevant_settings(nil, %User{} = user) do
[get_global_setting(), get_setting_for_user(user)]
|> Enum.reject(&is_nil(&1))
end
def load_relevant_settings(%Forum{} = forum, %User{} = user) do
[get_global_setting(), get_setting_for_forum(forum), get_setting_for_user(user)]
|> Enum.reject(&is_nil(&1))
end
I can’t figure out what’s going wrong. Has anybody an idea?
Edit: I already tried to set the pool size to a very high value (100), but it didn’t help.
Best regards,
CK
Most Liked
benwilson512
You can do:
def get_setting_for_forum(%Forum{} = forum) do
caller = self()
Cachex.fetch!(:cforum, "settings/forums/#{forum.forum_id}", fn ->
from(setting in Setting, where: setting.forum_id == ^forum.forum_id)
|> Repo.one(caller: caller)
end)
end
benwilson512
al2o3cr
One gotcha with this - AFAIK there’s no way to “un-allow” a connection, and Ecto gets grumpy if you call allow with a second PID. This makes for intermittently-failing tests when there’s a worker pool, as you’ll only get an error when a worker gets reused.
We ran into this issue with :poolboy; our fix was to have workers return :stop in tests so they don’t get re-used. This works, EXCEPT for very occasionally when there’s already a process waiting for a pool worker when the call returns; a fast-path optimization in Poolboy hands the now-exiting worker PID to the waiting process, which then fails 
lasseebert
Thanks @benwilson512, but I already use :shared mode on the connection, so it shouldn’t make a difference. (I also tried and it didn’t).
I don’t get the cannot find ownership process error, which would be solved with allow/3 or :shared, but a connection not available.
For now, I “solved” the issue by replacing the db-call inside the existing transaction with a mock that does not hit the db.
Here is a small example that reproduces my issue:
test "calling db from another process inside a transaction" do
Repo.transaction(fn ->
parent = self()
spawn_link(fn ->
User |> Repo.one()
send(parent, :done_in_other_process)
end)
receive do
:done_in_other_process -> :ok
end
:ok
end)
end
This is the setup code that is run before each db test:
:ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
It will give this error:
1) test calling db from another process inside a transaction (MyApp.MyTest)
test/my_test.exs:153
** (EXIT from #PID<0.497.0>) an exception was raised:
** (DBConnection.ConnectionError) connection not available and request was dropped from queue after 989ms. You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
(ecto_sql) lib/ecto/adapters/sql.ex:605: Ecto.Adapters.SQL.raise_sql_call_error/1
(ecto_sql) lib/ecto/adapters/sql.ex:538: Ecto.Adapters.SQL.execute/5
(ecto) lib/ecto/repo/queryable.ex:153: Ecto.Repo.Queryable.execute/4
(ecto) lib/ecto/repo/queryable.ex:18: Ecto.Repo.Queryable.all/3
(ecto) lib/ecto/repo/queryable.ex:67: Ecto.Repo.Queryable.one/3
test/my_test.exs:158: anonymous fn/1 in MyApp.MyTest."test calling db from another process inside a transaction"/1







