urbullet

urbullet

Preloading top comments for posts in Ecto using window functions

Note: I am using MySQL.

I have two tables named Post, Comment

  schema "posts" do
    field: id, string
    field :title, :string
    field :slug, :string
    field :active, :boolean, default: true
    timestamps()
  end

  schema "comments" do
    field: id, string
    field: post_id, string
    field :title, :string
    field :body, :string
    field :email, :string
    field :meta, :map
    field :active, :boolean, default: false
    timestamps()
  end

I am using Ecto for the database queries in Elixir. Currently, I am having the issue of trying to get all the posts in the database, and the most recent comment for those posts, if available. E.g:

Post 1 ──── post1__last_comment_id 
         
Post 2 ──── post2__last_comment_id      

Post 3 ──── nil

I have tried join_left on post.id == comment.id, but that returns all the comments related to the post not just the most recent one.

I tried also grouping_by comment.post_id and get the max id, while that worked, that is not the desired solution as there is a requirement for it to be based on date rather than id of the comment.

One more thing I tried after some suggestions on StackOverflow is using a window function since I cannot use a lateral join in MySQL. I tried to write an Ecto query for that and ended up with this:

Repo.all from p in Post, as: :post,
           join: c in assoc(p, :comments),
           inner_join: latest in subquery(
            from c1 in Comment, 
			select: {c1.start_date, over(row_number(), :parent)},
			windows: [parent: [partition_by: c1.post_id]]	
           ), on: latest.id == c.id,
           preload: [comments: c]

Unfortunately, that did not work and I am receiving this MySQL error but I can’t understand how to fix it:

** (Ecto.SubQueryError) the following exception happened when compiling a subquery.

** (Ecto.QueryError) subquery/cte must select a source (t), a field (t.field) or a map, got: `{&0.start_date(), over(row_number(), :department)}` in query:

from c0 in MyProject.Comments.Comment,
  windows: [parent: [partition_by: [c0.post_id]]],
  select: {c0.start_date, over(row_number(), :parent)}

Is it possible to be done using Ecto?

Marked As Solved

al2o3cr

al2o3cr

If you can’t upgrade to a version that supports window functions, other options:

  • write a query (probably with subqueries somehow?) that emulates what window functions do. This will probably be complicated and/or slow - that’s why window functions are useful.

  • reshape your data to make the query cheap. If your system has the same kind of expected usage pattern (lots of reads, relatively few writes, few/no deletes) as posts + comments, you could add a belongs_to :last_comment, Comment to Post. The code that creates a new comment would also update the post’s last_comment_id column.

Also Liked

al2o3cr

al2o3cr

The error message is complaining about this being a tuple; Ecto can’t use that for a subquery. Try a map instead: %{start_date: c1.start_date, ...}

Where Next?

Popular in Questions Top

LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement