jimmyhuang

jimmyhuang

How to do pagination in a nested graphql query with dataloader batch load?

I have a GraphQL server implemented with absinthe, dataloader and phoenix.

A sample query is as follows
What I anticipate is that the query lists the first 10 books in database and for each book, the first 10 chapters of the book.

{
  listBook(from:0, size:10){
    books {
      id
      name
      url
      chapters(from: 0, size: 10) {
        id
        name
        book{
          id
          name
          book {
            id
          }
        }
      }
    }
    total
  }
}

I am using dataloader to load chapters field.

However, the generated sql is not expected, instead of returning first 10 chapters for each book, the underlying sql actually returns first 10 chapters of all chapters of the first 10 books.

How can I achieve my goal? Make my GraphQL API return first 10 chapters of each book.

[debug] QUERY OK source="books" db=60.4ms
SELECT b0."id", b0."html", b0."info", b0."name", b0."preface", b0."text", b0."url", b0."info_html", b0."preface_html", b0."parent_id", b0."inserted_at", b0."updated_at", b0."parent_id" FROM "books" AS b0 WHERE (b0."parent_id" = ANY($1)) ORDER BY b0."parent_id" LIMIT $2 OFFSET $3 [[291289, 291066, 291009, 290794, 290647, 290409, 290196, 157109, 155130, 153693], 10, 0]

If I do not use dataloader but use a custom resolver, I can achieve my goal but that results in 1+10 queries because for each book the application needs to query database for its chapters.

Is there a way to use dataloader to batch load chapters?

The complete source code is here https://github.com/huangjimmy/wikisource

Most Liked

jimmyhuang

jimmyhuang

I come up with a solution and it does work. I do use PostgreSQL window function. The solution will only work with PostgreSQL.

1 I add a virtual field chapter_number

lib/wikisource/book.ex

schema "books" do
    field :chapter_number, :integer, virtual: true

2 I inject a chapters: true parameter so that datasource.ex knows a query is a chapter query

lib/wikisource_web/graphql/schema/types.ex

    field :chapters, list_of(:book) do
      arg(:offset, non_null(:integer))
      arg(:first, non_null(:integer))
      resolve(dataloader(DataSource, :chapters, args: %{chapters: true}))

3 I construct the query with window function

I only select a subset of fields because some field is too large and costly to load.

lib/wikisource_web/graphql/datasource.ex

  def query(queryable, params) do
   case params do
      %{chapters: true, offset: offset, first: first} ->
        last = offset + first
        query = from r in queryable, select: r, select_merge: %{chapter_number: fragment("row_number() over (PARTITION by parent_id order by \"name\")")}
        from r in subquery(query), select: %Wikisource.Book{id: r.id, name: r.name, info: r.info, preface: r.preface, info_html: r.info_html, preface_html: r.preface_html}, where: r.chapter_number >= ^offset and r.chapter_number < ^last
      %{order_by: order_by, offset: from, first: size} -> from record in queryable, order_by: ^order_by, offset: ^from, limit: ^size

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New

Other popular topics Top

bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement