silverdr

silverdr

Ecto - embeds_many as single map with unique keys

I have a Parent model, which embeds_many() children models. Children are then stored in a jsonb column as a List (Array in JSON) of simple, two-key maps (Objects), where values of one of the keys of those maps need to be unique. Something like:

[
	{"name":"name0", "amount":"10.00"},
	{"name":"name1", "amount":"12.20"},
	{"name":"name2", "amount":"10.00"},
	[…]
	{"name":"nameN", "amount":"56.00"}
]

Value of name has to be unique. I handled the uniqueness on create/update and this works. But then I continuously spend cycles all around the application converting the retrieved list into a single map like:

%{
	"name0" => "10.00",
	"name1" => "12.20",
	"name2" => "10.00",
	[…]
	"nameN" => "56.00"
}

Any suggestions for a “proper” way to get the data stored as a single map (JSON object), while retaining the possibility to use Phoenix form helpers / inputs_for, validations and other goodies?

Custom Ecto type? Or? Some examples, maybe?

Most Liked

LostKobrakai

LostKobrakai

Something like this:

defmodule MapFromArray do
  use Ecto.Type

  def type, do: {:array, :map}

  def load(data) do
    with {:ok, list} <- Ecto.Type.load(type(), data) do
      {:ok, Map.new(list, &{&1["name"], &1["amount"]})}
    end
  end

  def embed_as(_), do: :dump

  […]
end

defmodule ReadParent do
  use Ecto.Schema

  schema "parents" do
    field :children, MapFromArray
  end
end
  
  
Repo.insert_all("parents", [
  %{children: [%{name: "abc", amount: 2}, %{name: "def", amount: 5}]}
])

[%{children: %{"abc" => 2, "def" => 5}}] = Repo.all(ReadParent)

Could even consider Ecto.ParameterizedType if you don’t like hardcoding key/value keys.

LostKobrakai

LostKobrakai

This is full of caveats:

  • inputs_for only works for related assocs or embeds, which are one or a list of related records. There’s nothing like a map for related records.
  • The :map or {:map, value_type} native types of ecto are not supported by phoenix form handling natively. You can make things work by manually creating input names and assigning values to inputs, but it’s not covered ootb. Also using an ecto type over a relationship means you forego a bunch of their features like e.g. per item errors, sort/drop support, likely a few more…
  • How do you even model a “map” using form inputs in an abstract manner in the first place. There’s surely many specific examples of how to do that, but I’m not sure there’s a generic version phoenix could implement.

So really I’d treat the data like you do and let the goal of it becoming a map be an implementation detail. You could consider separating the “write model” from the “read model” here and let the read model read the data from the db in the correct format / into a :map field.

MarcusRiemer

MarcusRiemer

I got this to work recursively, but I think I have a more precise question now: Is there some public Ecto API to dump a whole struct?

When writing testcases for nested datasets I figured out that Ecto probably uses something along the lines of Ecto.Type.dump(MyModule.__schema__(:type, :field))to dump individual fields. So I’m now doing the same when dumping map values. The new implementation looks like this:

  def dump(data, dumper, params) when is_map(data) do
    {:ok, Map.values(data) |> Enum.map(& dump_struct(&1, dumper, params))}
  end

  # Checks whether any of the fields on the given struct are an embedded
  # map themselves. If this is the case the field is dumped with the type
  # information attached to the schema.
  defp dump_struct(data, dumper, params) when is_struct(data, params.value) do
    keys = data |> Map.from_struct() |> Map.keys()
    Enum.reduce(keys, data, fn key, data ->
      case params.value.__schema__(:type, key) do
        {:parameterized, {EctoSupport.EmbeddedMap, _params}} = t ->
          {:ok, transformed} = dumper.(t, get_in(data, [Access.key!(key)]))
          put_in(data, [Access.key!(key)], transformed)
        _ ->
          data
      end
    end)
  end

This gives me recursion on my own types and has so far passed my synthetic and practical tests. But if feels weird as I …

  • enumerate over all fields instead of somehow specifically grabbing those with matching types …
  • needed to jump through the Access.key! hoops to do the manipulation on the struct on the fly …
  • and disregard every field that is not a EmbeddedMap.

So there is probably a nicer way to do this.

Where Next?

Popular in Questions Top

yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
Kagamiiiii
Student &amp; New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

Other popular topics Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

We're in Beta

About us Mission Statement