elixirnewbie

elixirnewbie

Bulk inserts with Postgrex

What’s the best way to do bulk inserts with Postgrex?
If I have a list [[1,2],[3,4],[5,6]] what’s the ideal way to handle it in Postgrex.

Most Liked

OvermindDL1

OvermindDL1

Wooo, quite a number, I’d probably use the PostgreSQL COPY command.

Some relevant threads:

elixirnewbie

elixirnewbie

@benwilson512 I am using Postgrex.I know with Ecto you can do Repo.insert_all .I am trying to insert 50,000 records.So, till now I have
1.50000 |> Stream.map(fn(item) -> [item,item * 2] end) |> Stream.chunk(1000,1000,[]) |> Task.async_stream(fn(list) -> list |> Enum.each(fn(record) -> Postgrex.query!(pid, "INSERT INTO mytable(val1,val2) VALUES($1,$2) ", list) end) end)

It works but I am trying to insert multiple values in a single transaction

hubertlepicki

hubertlepicki

So Postgrex can insert multiple records, and this is precisely what Ecto is doing. If you use Ecto’s insert_all, it inserts multiple records using single INSERT command like this:

INSERT INTO "table" ("a","b","c") VALUES ($1,$2,$3),($4,$5,$6), ... ,($97,$98,$99) ["A", "B", "C", "A", "B", "C", ... , "A", "B", "C"]

If you are using Postgrex directly, I think you need to assemble that SQL on your own, I don’t think it gives you any shortcuts.

Where Next?

Popular in Questions Top

aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement