Qqwy

Qqwy

TypeCheck Core Team

When to use Ecto , when to use Mnesia

I want to create a simple application (But one that is not tightly coupled with Phoenix). One of the things I am considering, is whether it would be better to use Ecto (which right now does mean that an external database like Postgres or MySQL is necessary), or Mnesia:

  • I really like Ecto’s query language.
  • I really like not being dependent on external services; keeping everything inside OTP-land.

What are the considerations I should make? Why should I pick one over the other?

Most Liked

wojtekmach

wojtekmach

Hex Core Team

I also recommend just sticking with Postgres, at least for now.

What would be cool, though, is if there would be an Ecto<->Mnesia adapter at some point, because being completely independent from external services remains very compelling.

Looks like someone already started writing an adapter but it’s not ready for production yet: GitHub - webengineer-max/mnesia_ecto: Ecto adapter for Mnesia.

I can wholeheartedly recommend trying to write your own adapter as a weekend project or something - I learned a lot about Ecto’s internals (and appreciated it’s design) by doing so.

Shameless plug: you can look at my experiments at github_ecto or ets_ecto. A bare minimum adapter that does something could be this: lib/ets_ecto.ex (from an early commit in that project)

13
Post #7
cmkarlsson

cmkarlsson

Yes, one of the downsides:)

  • You need to manually handle split-brain situations. There are a few well known tricks to do this with various trade-offs but at then end your application will have to handle it and decide what to do.
  • You need to keep all your data in memory. Well, you could use dets but then you have a 2GB restriction on table-size. Also there are new backends for mnesia which may solve this (level db)
  • You can use fragmented tables to handle above but your application must set this up and handle the fragments.
  • It is prone to get overloaded and handling it in distributed settings is harder than you think.
  • Slow start-up times (depending on order nodes are started mnesia may decide to copy entire tables to different nodes, this slows down startup time significantly)
  • Upgrading table definitions may be hard in a distributed setting and without stopping access to the database.

All this said. Mnesia is a very battle-tested database, it is just that it requires a fair bit of skill to operate and that you know its limitation. There are some pretty big shops running mnesia as a major part of their infrastructure (Whatsapp, Klarna).

Why would you want to use mnesia?:

  • You can’t get a faster database for reads as it runs in the same memory space as your erlang application. Anything else handles reads under 40 micro-seconds?
  • No impedance mismatch between erlang code and the database. You read terms straight of memory so no serialization layers and no database traffic
  • It has transactions. They are of course more limited than a proper database but not many other NoSQL databases have such good history
  • Distribution. Have its own problems of course but it is a distributed database :slight_smile:

Mnesia isn’t a replacement for postgresql though, It is not an RDMS and as with any other NoSQL database you must have a use case where it fits.

OvermindDL1

OvermindDL1

Just to put my bit in:

I use PostgreSQL for anything and everything that needs to persist.
If postgresql is slow on certain queries that I cannot really make better than I either make a materialized view inside postgresql that updates on occasion, or if I do a ton of tiny little queries per transaction that all are the same (like getting permissions across connections, websockets, etc…) then I use CacheX (with a fallback in it set to querying the database) as it has auto-timeout and removal, janitor process, and far far more, including transactions and shared retrievals (it is built on ETS of course).

AstonJ

AstonJ

This is a good FAQ on Mnesia - any help?

http://erlang.org/faq/mnesia.html

11 Mnesia Database Questions

Mnesia is the somewhat odd name for the real-time, distributed database which comes with Erlang.

11.1 What is Mnesia good at?

Locking and Transactions

If you need to keep a database that will be used by multiple processes and/or nodes, using Mnesia means you don’t have to write your own access controls.

Distribution

Tables can be replicated at many nodes, both for efficiency (database lookup is a local activity) and robustness (redundancy means if one node goes down, other nodes still have copies of the data.)

Non-fully-normalised data

Unlike most database systems, records can contain data of arbitrary size and structure.

Monitoring

Monitoring - processes can subscribe to events which are sent when various operations on the data take place (update, delete, etc) The RDBMS package allows even more fine-grained control.

11.2 What is Mnesia not so good at?

Mnesia is primarily intended to be a memory-resident database. Some of its design tradeoffs reflect this.

Really large tables must be stored in a fragmented manner.

bentanweihao

bentanweihao

Author of The Little Elixir and OTP Guidebook

Then I think you can go with something like https://github.com/meh/amnesia. It’s querying syntax is pretty nice. But I would consider Ecto + Postgres an essential, well supported and documented dependency.

Where Next?

Popular in Questions Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

We're in Beta

About us Mission Statement