Harrygr

Harrygr

Batch loading resources in Absinthe using dataloader for a non-ecto struct

In my Phoenix/Absinthe app I have an object in my graphql schema like this

object :vote_result do
  field(:id, non_null(:id))
  field(:score, non_null(:integer))
  field(:top_upvoted_user, non_null(:user))
end

I have a query whose purpose is to return a list of vote_results. vote_result is not part of my database schema and is queried from a non-trivial ecto query.

schema:

field :vote_results, type: non_null(list_of(:vote_result)) do
  resolve(&VoteResolver.get_all/3)
end

resolver:

def get_all(_,_,_) do
   results = # non-trivial ecto query
   {:ok, results}
end

the results from my ecto query look like this:

[
  %{
    id: 1,
    score: 5,
    user_id: 14,
  },
  %{
    id: 2,
    score: 8,
    user_id: 34,
  }
]

I.e. they are simple the result of the query - they don’t correspond to any part of my schema.

I’d like to resolve a full user for the purpose of the vote_result. The naive way would to just add a resolution function to my :vote_result object:

object :vote_result do
    field(:top_upvoted_user, non_null(:user), resolve: fn _, %{source: result} -> Repo.get(User, result.user_id end)
end

however this results in the n+1 query problem where a separate SQL query needs to be performed to resolve each user in the list.

I’m already using dataloader to batch resolve nested Ecto relations but I’m not sure how to use it to resolve objects from a non-ecto source (i.e. just a map containing an ID of an ecto resource).

Can anyone steer me in the right direction? I’ve tried something like the following to no avail:

defmodule MyApp.DataLoader do
  alias MyApp.User
  import Ecto.Query

  def data() do
    Dataloader.Ecto.new(MyApp.Repo, run_batch: &run_batch/5)
  end

  def run_batch(_, _query, :top_upvoted_user, vote_results, repo_opts) do
    user_ids = Enum.map(vote_results, & &1.user_id)

    result =
      from(u in User,
        where: u.id in ^user_ids
      )
      |> MyApp.Repo.all(repo_opts)
      |> Map.new()

    Enum.map(user_ids, fn id -> [Map.get(result, id)] end)
  end

  # Fallback to original run_batch
  def run_batch(queryable, query, col, inputs, repo_opts) do
    Dataloader.Ecto.run_batch(MyApp.Repo, queryable, query, col, inputs, repo_opts)
  end
end

I get the following error:

Request: POST /graphql
** (exit) an exception was raised:
** (Dataloader.GetError) The given atom - :top_upvoted_user - is not a module.

This can happen if you intend to pass an Ecto struct in your call to
dataloader/4 but pass something other than a struct.

Most Liked

Harrygr

Harrygr

I seem to have managed to solve this by using the Dataloader.KV source.

First, I defined a dataloader function to batch load my users:

defmodule MyApp.VoteResultDataLoader do
  def load({:top_upvoted_user, _}, vote_results) do
    users =
      vote_results
      |> Enum.map(& &1.user_id)
      |> get_user_list()

    vote_results
    |> Map.new(fn vr -> {vr, Map.get(users, vr.user_id)} end)
  end
end

One thing that wasn’t obvious or well documented is that load/2 must return a Map whose keys are the original parent entities passed to the function. E.g.

%{
  %{id: 1, score: 5, user_id: 14} => %MyApp.User{id: 123, ...etc}
}

Once I have my dataloader function I can add it as a source to my schema’s dataloader:

Dataloader.add_source(
        :vote_results,
        Dataloader.KV.new(&MyApp.VoteResultDataLoader.load/2)
      )

Finally, in my object resolution I just use the dataloader/1 function:

object :vote_result do
    field(:top_upvoted_user, non_null(:user), resolve: dataloader(:vote_results))
end

I’m not sure if I’ve arranged everything as well as it can be. I’m never that sure where to put this dataloader logic and to what part of my app it belongs in. But this will depend on one’s own application.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

Other popular topics Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement