maxim

maxim

Another cast vs dump question for a custom Ecto type

Hi folks,

I’ve written custom types before, but this specific case gives me pause, and I wonder if you could explain your reasoning here when deciding how to split this logic between cast and dump.

I have a list of base-3 integers that I store in the database as a :binary (postgres bytea).

The value that I assign looks like this (up to 127 digits):

[0,2,1,0,2,2,1]

The value database stores looks like this

[0,2,1,0,2,2,1] |> Enum.reverse |> Integer.undigits(3) |> :binary.encode_unsigned
# <<5, 112>>

However, I have to run 3 checks:

  1. Is this a list?
  2. Is length in 0..127
  3. Are all digits in 0..2?

What part of this goes into cast, and what goes into dump?

Here’s what I think, correct me if I’m wrong:

  • Cast runs the above 3 checks, but keeps it a list.
  • Dump ALSO runs the 3 checks (in case value was assigned without cast) but converts it into a binary.

However, those 3 things are quite a bit of code. Between 2 guards and an Enum.any? they traverse the whole list a couple of times. If this is correct, am I supposed ignore the fact that I’m running the same logic twice, once for cast and once for dump?

Marked As Solved

LostKobrakai

LostKobrakai

Ecto.Types handle 3 different data representations. Outside data (most often from form inputs, or json fields in the db), runtime data (how your data is put into your structs by ecto and used at runtime) and the database representations as db column.

cast goes from outside data -> runtime data
dump goes from anything (outside data or runtime data) -> database column data
load goes from database column data -> runtime data

For json (map) fields there’s also
json encoding from anything -> database json value
cast from database json value -> runtime value

The reason why cast/json encoding need to deal with “anything” is because there’s nobody stopping you from inserting data via Repo.insert_all or manipulating a struct manually and there’s no casting involved. So any validation of correct data needs to be done in both cast and dump, where one converts to runtime data and the other to database column data.

Also Liked

axelson

axelson

Scenic Core Team

I like visuals so I made a visual to represent @LostKobrakai’s awesome explanation:

And one thing that sometimes confuses (me at least) is that sometimes the “Outside data” and “Runtime data” have the same representation, for example for Ecto.UUID. Seeing it in this visual form makes the reason behind that easier for me to understand/remember:

10
Post #7
LostKobrakai

LostKobrakai

Already merged:

NobbZ

NobbZ

I’m not sure where to put the checks, but at least I can tell you that you only need to iterate once, roughly like this:

def valid_custom_type?(list) do
  list
  |> Enum.reduce_while(0, fn
    _d, l when l > 127 ->
      {:halt, :invalid}
    d, _l when d not in 0..2 ->
      {:halt, :invalid}
    _d, l ->
      {:cont, l + 1}
  end)
  |> is_integer()
end

This will iterate not more than once and not more than the first 128 digits of the list.

Due to the early return it should not become a bottleneck that soon.

maxim

maxim

That’s a great explanation, and from what I gather, for the question of “Should we be ok with doing the same work twice, once in cast and once in dump” the answer is “yes”.

And since outside data can also be formatted consistently with runtime data, would it be more correct to say that

cast goes from anything to runtime data
dump goes from anything to database column data

With this in mind, it’s almost always ok to actually call cast function from your dump function, does this sound right? (I know it depends, but I’ve noticed I could do that in most custom types I wrote).

ream88

ream88

Omg, can this answer please be included in the official Elixir docs? Maybe as a cheatsheet. I’ve never read a better explanation than this one! :blush:

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement