dli

dli

Support :insert_all in data-modifying CTE / WITH statements

Ecto currently supports some data-modifying WITH statements / CTEs for Postgres:

Options: […]
:operation - one of :all , :update_all , or :delete_all indicating the operation type of the CTE query. If blank, it defaults to :all , making the CTE query a SELECT query.
[…]
For Postgres built-in adapter, it is possible to define data-modifying CTE queries:

update_categories_query =
  Category
  |> where([c], is_nil(c.parent_id))
  |> update([c], set: [name: "Root category"])
  |> select([c], c)

{"update_categories", Category}
|> with_cte("update_categories", as: ^update_categories_query, operation: :update_all)
|> select([c], c)

Postgres, however, also supports INSERT statements inside CTEs:

-- EXAMPLE
with expensive_calc as (
   select customer_id, sum(amount) as total 
   from orders 
   group by customer_id
),
new_segments as (
   insert into segments (customer_id, total)
   select customer_id, total 
   from expensive_calc
   returning id, customer_id
)
insert into segment_history (segment_id, customer_id)
select id, customer_id 
from new_segments;

This allows multiple INSERTs to access the same source data, as well as the newly inserted data without crossing the database boundary.

The closest solutions in Ecto are:

  1. Create temp table for expensive_calc (needs fragment)
  2. Repeat expensive_calc as subquery (likely does the calculation twice)

… then use two sequential Repo.insert_all statements. However, this needs serialization of the RETURNING data and unnecessarily crosses the database ↔︎ app boundary.

Other examples:

Ecto syntax

I assume this would require a new macro insert/3 similar to update/3 which allows adding returning: [p.id, p.title, ...]? Haven’t given this too much thought.

Judging by the ecto_sql source and associated PR, this was initially planned but not implemented yet:

Maybe now is the right time?

First Post!

dli

dli

Bump to gather comments from top contributors — wdyt @josevalim @ericmj @michalmuskala?

Where Next?

Popular in Proposals: Ideas Top

MeerKatDev
many times we do stuff like (e.g. test setups, json views in phoenix) aaa = ... bbb = ... ccc = ... %{aaa: aaa, bbb: bbb, ccc: ccc} and...
New
rhcarvalho
Hi all, I would like to gather some feedback before a more intentional proposal to add a new :depth option when specifying a Git depende...
New
rekkice
I’m building an editor integration to evaluate Elixir code in an IEx session. While Code.eval_string/3 allows tracking variable bindings,...
New
kip
Sumary of proposal DateTime.from_iso8601/3 adjusted to: set all numeric fields to the values as parsed (not shifted to UTC), preservi...
New
pdgonzalez872
Hi! There has been some discussion about hiring/jobs on here and I thought about running this by everyone. I wanted to try to help recr...
New
dli
Ecto currently supports some data-modifying WITH statements / CTEs for Postgres: Options: […] :operation - one of :all , :update_all ,...
New
winsalva
Hello all. First of all i’m running this using termux on an android phone. Running mix assets.setup shows this message 06:54:08.450 [de...
New
bartblast
This could resolve to {[a: 1, b: 2]}. Was it ever considered to allow such syntax? Notice this: {:abc, a: 1, b: 2} and this: my_fun(:abc,...
New
sezaru
When writing my code, I always find __MODULE__ very useful to use as alias of that module “inner dependencies”, ex: alias __MODULE__.{Im...
New
dkuku
This is a proposal to make the map key mismatch errors a bit better: Every time I have a typo It’s very challenging for me even when I u...
New

Other popular topics Top

josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

We're in Beta

About us Mission Statement