astjohn

astjohn

Help with subquery including joins and preloads for pagination

Hi All,

I’m trying to perform the following SQL query:

SELECT t.*, d.*, u.* FROM 
(
    SELECT x.* FROM trips AS x
    INNER JOIN trips_companies AS tc ON tc.trip_id = x.id
    INNER JOIN companies AS c ON tc.company_id = c.id    
    WHERE c.id = '91955970-34eb-4a1b-97e6-53e9084abbfa'
    ORDER BY x.start DESC
    LIMIT 100 
    OFFSET 0
) AS t
INNER JOIN destinations AS d ON d.trip_id = t.id
LEFT JOIN trips_travellers AS tt ON tt.trip_id = t.id
LEFT JOIN users AS u ON tt.user_id = u.id
ORDER BY t.start DESC

My first attempt was the following:

page_number = 1
page_size = 100
offset = page_size * (page_number - 1)

sub = Trip
      |> join(:inner, [t], c in assoc(t, :companies))
      |> where([_t, c], c.id == ^company.id)
      |> order_by([t, _c], desc: t.start)
      |> limit(^page_size)
      |> offset(^offset)

query = subquery(sub)
        |> join(:inner, [t], d in assoc(t, :destinations))
        |> join(:left, [t, _d], u in assoc(t, :travellers))
        |> order_by([t, _d, _u], desc: t.start)
        |> preload([_t, d, u], destinations: d, travellers: u)

I’ve tried a bunch of combinations without any luck. As soon as the preload is introduced, an error is thrown. The only thing I haven’t tried is the use of fragments. Is there a way to do it without fragments?

Thanks!

Most Liked

astjohn

astjohn

@Ankhers - The error is:

** (MatchError) no match of right hand side value: %Ecto.SubQuery{cache: [:all, 0, {:offset, {:^, [], [0]}}, {:limit, {:^, [], [0]}}, {:order_by, [[desc: {{:., [], [{:&, [], [0]}, :start]}, [], []}]]}, {:where, [and: {:==, [], [{{:., [], [{:&, [], [1]}, :id]}, [], []}, {:^, [], [0]}]}]}, {:join, [{:inner, {"trips_companies", nil}, {:==, [], [{{:., [], [{:&, [], [2]}, :trip_id]}, [], []}, {{:., [], [{:&, [], [0]}, :id]}, [], []}]}}, {:inner, {"companies", Radar.Accounts.Company, 33029540}, {:==, [], [{{:., [], [{:&, [], [2]}, :company_id]}, [], []}, {{:., [], [{:&, [], [1]}, :id]}, [], []}]}}]}, {"trips", Radar.Trips.Trip, 50918967}], params: [<<145, 149, 89, 112, 52, 235, 74, 27, 151, 230, 83, 233, 8, 74, 187, 250>>, 100, 0], query: #Ecto.Query<from t0 in Radar.Trips.Trip, join: c in "trips_companies", on: c.trip_id == t0.id, join: t1 in Radar.Accounts.Company, on: c.company_id == t1.id, where: t1.id == ^"91955970-34eb-4a1b-97e6-53e9084abbfa", order_by: [desc: t0.start], limit: ^100, offset: ^0, select: %Radar.Trips.Trip{id: t0.id, mongo_id: t0.mongo_id, type: t0.type, name: t0.name, start: t0.start, finish: t0.finish, read: t0.read, muted: t0.muted, trip_type: t0.trip_type, activities: t0.activities, company_name: t0.company_name, employee_name: t0.employee_name, final: t0.final, deleted_at: t0.deleted_at, settings: t0.settings, created_at: t0.created_at, updated_at: t0.updated_at}>, select: {:struct, Radar.Trips.Trip, [id: {:value, :binary_id}, mongo_id: {:value, :string}, type: {:value, :integer}, name: {:value, :string}, start: {:value, :utc_datetime}, finish: {:value, :utc_datetime}, read: {:value, :boolean}, muted: {:value, :boolean}, trip_type: {:value, :integer}, activities: {:value, {:array, :integer}}, company_name: {:value, :string}, employee_name: {:value, :string}, final: {:value, :boolean}, deleted_at: {:value, :utc_datetime}, settings: {:value, :map}, created_at: {:value, :utc_datetime}, updated_at: {:value, :utc_datetime}]}}
    (ecto) lib/ecto/repo/queryable.ex:124: Ecto.Repo.Queryable.execute/5
    (ecto) lib/ecto/repo/queryable.ex:37: Ecto.Repo.Queryable.all/4

@idi527 - With select instead of preloads, it seemed to work at first. However, the number of results returned was higher than the limit and so likely the joined rows were still included.

We also tried the following:

sub = Trip 
      |> join(:inner, [t], c in assoc(t, :companies)) 
      |> where([_t, c], c.id == ^company.id) 
      |> order_by([t, _c], desc: t.start) 
      |> limit(^page_size) 
      |> offset(^offset)
query = Trip 
        |> join(:inner, [t], st in subquery(sub), t.id == st.id) 
        |> join(:inner, [t, _st], d in assoc(t, :destinations)) 
        |> join(:left, [t, _st, _d], u in assoc(t, :travellers)) 
        |> order_by([t, _st, _d, _u], desc: t.start) 
        |> preload([_t, _st, d, u], destinations: d, travellers: u), 
        |> select([t, _st, _d, _u], t)

Which resulted in the number of results being slightly lower that the limit. I’ve yet to find a proper way to do this in one shot via subqueries and joins. Thoughts?

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

We're in Beta

About us Mission Statement