wilkinson4

wilkinson4

Ecto Postgres select ARRAY constructor

Hello,

We recently did some optimizations on a slow query in our system, and found the following article from dba exchange that has a nice syntax for selecting an array of items per row

This ended up being a more performant solution from what we were using previously. Below is a simple example of what we’re trying to do in raw postgresql

select *
from calls c
  left join lateral (
    select array (
        select cr.id
        from call_recordings cr
        where cr.call_id = c.id
          and cr.s3_key is not null) as call_recording_ids) as call_recordings on true
limit 10;

As far as I can tell there isn’t an easy way to do this in Ecto. The closest translation to Ecto I’ve got so far is this

  def test_select_array do
    call_recordings_query =
      from cr in Recording,
        select: %{
          call_recording_ids:
            fragment(
              "(SELECT ARRAY(SELECT cr.id FROM call_recordings cr WHERE cr.call_id = ? AND cr.s3_key IS NOT NULL))",
              parent_as(:call).id
            )
        }

    from(c in Call,
      as: :call,
      left_lateral_join: cr in subquery(call_recordings_query),
      limit: 10,
      select: %{
        call_id: c.id,
        call_recording_ids: cr
      }
    )
  end

But even that isn’t quite right, and it’s janky. Looking for any ideas/guidance on this. Maybe this is something Ecto supports without a fragment and I’m not aware of it, or could support?

– Will

Marked As Solved

LostKobrakai

LostKobrakai

import Ecto.Query

array_query = 
  from cr in "call_recordings",
    where: cr.call_id == parent_as(:call).id,
    where: not is_nil(cr.s3_key),
    select: %{
      id: cr.id
    }

call_recordings_query =
      from cr in subquery(array_query),
        select: %{
          call_recording_ids: fragment("array_agg(?)", cr.id)
        }
query = 
  from(c in "calls",
    as: :call,
    left_lateral_join: cr in subquery(call_recordings_query), on: true,
    limit: 10,
    select: %{
      call_id: c.id,
      call_recording_ids: cr
    }
  )

{sql, _} = Repo.to_sql(:all, query)
IO.puts(sql)
# SELECT c0."id", s1."call_recording_ids" 
# FROM "calls" AS c0 
# LEFT OUTER JOIN LATERAL (
#   SELECT array_agg(ss0."id") AS "call_recording_ids" 
#   FROM (
#     SELECT ssc0."id" AS "id" 
#     FROM "call_recordings" AS ssc0 
#     WHERE (ssc0."call_id" = c0."id") AND (NOT (ssc0."s3_key" IS NULL))
#   ) AS ss0
# ) AS s1 ON TRUE 
# LIMIT 10

Functionally this should yield the same result as the requested query

Also Liked

D4no0

D4no0

You could move the fragment to a macro to make the query cleaner, here is an example on how that looks like:

defmacro st_transform(wkt, srid) do
  quote do: fragment("ST_Transform(?, ?)", unquote(wkt), unquote(srid))
end
Schultzer

Schultzer

This is where Ecto start to show it’s brittleness, as one of the best ORM I have ever worked with, it still falls short in a lot of real world cases and especially on a enterprise level where you usually have dedicated DBAs that don’t know Elixir, GitHub - elixir-dbvisor/sql: Brings an extensible SQL parser and sigil to Elixir, confidently write SQL with automatic parameterized queries. could help you out here, still fresh but the idea is simple, lower the barrier between Elixir and SQL.

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement