jeffdeville

jeffdeville

Arch Question: Tired of passing this same parameter around for every method call

Situation

I’m working on a multi-tenant library for Ecto. Ecto structs and queries let you set a prefix in the meta area, which is awesome. But it’s getting really verbose to send every query and changeset through a set_prefix() method before forwarding it along to the Repo.

Thoughts / Ideas

  1. This may simply be the elixir/functional way. After all, less magic is good, and not hiding your function inputs is good too. :slight_smile:

  2. Currying - Currying isn’t quite what I want because I’m currying a module, not a function. (Also, I’m not changing the number of parameters required)

  3. Wrapping - I have a repo wrapper that verifies that a prefix was set on relevant Ecto structs. This is easy because I’m just verifying that SOMETHING was set, which means I can build it at compile time with macros. But what I’d really like to be able to do is:

    repo = Repo.prefixify(123)
    user = User.changeset(%User{}, %{name: “joe”})
    repo.insert(user)

and have it sent to the 123 prefix. Automatically. Then later,

loaded_user = repo.get!(User, user.id)

and have it know to pull from prefix 123.

So repo itself is a module.

Agents seem like the closest strategy here. However, it seems like with an agent, you have to keep track of your agent’s pid. Then you call your regular module, just including the pid as an argument. But at this point, all I’ve done is traded passing in a prefix for a pid.

Is there any way to pass this in once, and be done with it? If not, no worries. I just don’t want to let my unfamiliarity with the language limit the easy of use of my library.

Marked As Solved

josevalim

josevalim

Creator of Elixir

One option is to use the process dictionary along side custom functions in the repository. So you would do:

defmodule MyApp.Repo do
  use Ecto.Repo, otp_app: :my_app

  def put_prefix(prefix) do
    Process.put({MyApp.Repo, :prefix}, prefix)
  end

  def tenant_get(query, opts) do
     get(query, Keyword.put_new(opts, :prefix, get_prefix())
  end

  defp get_prefix do
     Process.get({MyApp.Repo, :prefix}) || raise "no Repo prefix set"
  end
end

And then:

MyApp.Repo.tenant_get(User, id)

Keep in mind I am using Ecto 2.1 (currently out as a release candidate) ability to pass the prefix as an option to all Repo operations.

Also Liked

michalmuskala

michalmuskala

I’d say we missed the obvious solution here - anonymous functions.

If we were talking about a single function instead of whole repo - this would be obvious with using partial application, wouldn’t it?

prefixed_insert = &Repo.insert(&1, prefix: "foo")
prefixed_insert.(my_data)

So can we do something similar for a module? We need to do some changes, mainly because now we need to decide at runtime which function to call, fortunately we can use apply/3.

def prefixify(prefix) do
  fn fun, args ->
    [opts | rest] = Enum.reverse(args)
    apply(Repo, fun, Enum.reverse(rest, [Keyword.put_new(opts, :prefix, prefix)])
  end
end

This allows us to call:

prefixed = Repo.prefixify("foo")
prefixed.(:insert, [my_data, []])
prefixed.(:all, [some_query, []])

It’s a bit different than the original, but achieves the goal. Is it worth it and should be done? That’s a completely different question :wink:

jeffdeville

jeffdeville

Nov 2, 2016 Update

So indeed my tests were where I was feeling the most pain.

I worked out a solution in 2 parts for that.

  1. I’m running a test_seeds.exs script at the beginning of all of my specs to do standard setup. That’s reduced general duplication quite a bit, and sped the specs up tremendously as well (9 sec -> .8, because the specs were actually creating tenants)
  2. Where I am creating extra setup data, I was unable to use the Ecto strategy for ex_machina, because I couldn’t set the prefix on those structs. So I created a PR for ex_machina (ExMachine PR) that kinda feels like ‘traits’ from factory girl.

When I get back to this project, I’ll look into @josevalim’s insight about Ecto 2.1, and its ability to accept the prefix as a keyword to Repo operations. That is probably the ‘good enough’ solution right there!

Oct 31, 2016 Update

Update for future readers. I realized that the place I was noticing all of this painful duplication was in my specs. But my specs are usually only using a single prefix, and part of the struggle was with ex_machina needing a new (prefix) parameter that hosed its lovely strategy pattern. So the compromise solution I’m going with now is to create a test-only import file that will wrap ex_machina, letting me pass in a tenant that is defaulted to a ‘test’ tenant. And then also wrappers around the Repo that do the same. So far (this is a work in process), I’m not planning to use the wrappers in my production code, because my methods are short, and so I’m comfortable seeing the specification of the tenant. Hopefully, once this is is all complete, I’ll bake it in to the library docs.

That said, if there’s a solution to my original question that I’ve missed, I’d still really like to know!

pba

pba

You could use the pipe operqator |> and instead of:

do

repo = Repo.prefixify(123)
%User{}
|> User.changeset(%{name: "joe"})
|> repo.insert

Also if the module where you to this is User specific you might consider import User
EDIT: Expanded original answer
Regarding the prefix: How about using something like this:

defmodule User do
  defmacro __using__(prefix) do
    quote do
      use Ecto.Schema
      @schema_prefix prefix || Application.get_env(:ex_machina, :prefix, "public")
      #... all the other user stuff
  end
  end
end

and then

defmodule MyPrefixUser do
  use User :my_prefix
end
jeffdeville

jeffdeville

Thank you for the detailed writeup, @pba. I wasn’t as clear as I should have been on what I was trying to achieve. The problem I’m struggling with was how to write the function Repo.prefixify(123), such that it returned a version of the Repo that would apply the given prefix to either the query or changeset appropriately. The problem is that the prefix will change based on each request. I’m using postgresql’s schemas to isolate client data. As a result, I can’t set the prefix in a config. Poor explanation on my part. Thank you for the input!

OvermindDL1

OvermindDL1

Each request gets its own process. The Erlang scheduler ensures nothing shared between processes. There is no pool of processes (rather a pool of memory, it is low level stuff, just assume erlang does things right, because it does ^.^), no need to clear it unless you want it cleared for your current request for some reason. :slight_smile:

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

Other popular topics Top

sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

We're in Beta

About us Mission Statement