axelson

axelson

Scenic Core Team

A comprehensive approach to data loading and permissions

I am working on an application of mid to high size and complexity, it is an umbrella application with at least 3 main applications (but you could think of them as Phoenix Contexts if you wish). There is an existing client-facing JSON API, a little pure HTML rendering, some websockets, and an internal API (for another server to interface with us). Currently data-loading and permissions checking happens in an ad-hoc manner. As we are just starting to add GraphQL support with Absinthe I’ve been thinking of using Dataloader throughout the various applications and contexts. Does anyone have familiarity with this type of approach?

I have been fairly deeply impacted by the philosophy espoused by Dan Schafer in this GraphQL talk: https://www.youtube.com/watch?v=etax3aEe2dA

So my goals are to have entities always loaded with respect to the current viewer (usually a user) and for that data loading to live in the context of the specific application.

The main approach that has come to mind so far is to create a %UserRequest{} struct with a %Viewer{} struct embedded. The %Viewer{} would have typically have a normal %User{} associated with it but sometimes the user would be a :super_admin (i.e. me) which would have extra permissions and it might even be :other_internal_system. The %UserRequest{} struct would additionally contain the %Dataloader{} that could be used to fetch entities that the user is allowed to see.

So my context/application data fetching functions would look something like this:

defmodule Blog do
  @spec fetch_post(binary, %UserRequest{}) :: {:ok, {%Post{}, %UserRequest{}}} | {:error, term}
  def fetch_post(id, %UserRequest{loader: loader, viewer: viewer}) do
    ...
  end

  @spec fetch_posts(list(binary), %UserRequest{}) :: {:ok, {list(%Post{}), %UserRequest{}}} | {:error, term}
  def fetch_posts(ids, %UserRequest{loader: loader, viewer: viewer}) do
    ...
  end
end

So the context/application data fetching functions would take in the data to query, along with the %UserRequest{} (which now that I think of it should perhaps be called a %UserRequestContext{} or maybe just %RequestContext{}). And along with fetching and returning the actual data, they would potentially load new data into the %UserRequest{} data structure which is the reason that the %UserRequest{} needs to be returned from the data fetching method as well.

This would allow all of the data fetching methods to have a common format for permission checking and avoid loading the same data multiple times (note that the permission checking will sometimes have to load data specific to the user).

Does this approach make sense? Are there better alternatives that wouldn’t require re-writing so much of the data-fetching logic? Should I instead move the dataloader to the Process dictionary (which is effectively the approach taken by the javascript dataloader)? @jfrolich I’m especially interested in your thoughts since I think your talk (which I’m about to watch) and PR are related to this topic.

Most Liked

jfrolich

jfrolich

Definitely makes sense to me, in my application the contexts indeed take a user as the last argument for authorization.

It’s not really clear to me how including %UserRequestContext{} as the result of the function solves the dataloading/n+1 problem.

That is indeed exactly the problem I’m trying to solve with the PR. Interested what you think about the talk and the approach taken in the PR. Unfortunately I didn’t have much time to work on this. But help is welcome, I think it’s a pretty important element for more complex GraphQL apps.

PS: using the PR you don’t need to pass the loader, so you can just use the viewer as the last argument. Only in the graphql the deferrable is resolved using the dataloader. The deferrables are as much as possible resolved at the same time, so batching, caching and parallelization is applied.

Where Next?

Popular in Questions Top

bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement