Qqwy

Qqwy

TypeCheck Core Team

When to use Ecto , when to use Mnesia

I want to create a simple application (But one that is not tightly coupled with Phoenix). One of the things I am considering, is whether it would be better to use Ecto (which right now does mean that an external database like Postgres or MySQL is necessary), or Mnesia:

  • I really like Ecto’s query language.
  • I really like not being dependent on external services; keeping everything inside OTP-land.

What are the considerations I should make? Why should I pick one over the other?

Most Liked

wojtekmach

wojtekmach

Hex Core Team

I also recommend just sticking with Postgres, at least for now.

What would be cool, though, is if there would be an Ecto<->Mnesia adapter at some point, because being completely independent from external services remains very compelling.

Looks like someone already started writing an adapter but it’s not ready for production yet: GitHub - webengineer-max/mnesia_ecto: Ecto adapter for Mnesia.

I can wholeheartedly recommend trying to write your own adapter as a weekend project or something - I learned a lot about Ecto’s internals (and appreciated it’s design) by doing so.

Shameless plug: you can look at my experiments at github_ecto or ets_ecto. A bare minimum adapter that does something could be this: lib/ets_ecto.ex (from an early commit in that project)

13
Post #7
cmkarlsson

cmkarlsson

Yes, one of the downsides:)

  • You need to manually handle split-brain situations. There are a few well known tricks to do this with various trade-offs but at then end your application will have to handle it and decide what to do.
  • You need to keep all your data in memory. Well, you could use dets but then you have a 2GB restriction on table-size. Also there are new backends for mnesia which may solve this (level db)
  • You can use fragmented tables to handle above but your application must set this up and handle the fragments.
  • It is prone to get overloaded and handling it in distributed settings is harder than you think.
  • Slow start-up times (depending on order nodes are started mnesia may decide to copy entire tables to different nodes, this slows down startup time significantly)
  • Upgrading table definitions may be hard in a distributed setting and without stopping access to the database.

All this said. Mnesia is a very battle-tested database, it is just that it requires a fair bit of skill to operate and that you know its limitation. There are some pretty big shops running mnesia as a major part of their infrastructure (Whatsapp, Klarna).

Why would you want to use mnesia?:

  • You can’t get a faster database for reads as it runs in the same memory space as your erlang application. Anything else handles reads under 40 micro-seconds?
  • No impedance mismatch between erlang code and the database. You read terms straight of memory so no serialization layers and no database traffic
  • It has transactions. They are of course more limited than a proper database but not many other NoSQL databases have such good history
  • Distribution. Have its own problems of course but it is a distributed database :slight_smile:

Mnesia isn’t a replacement for postgresql though, It is not an RDMS and as with any other NoSQL database you must have a use case where it fits.

OvermindDL1

OvermindDL1

Just to put my bit in:

I use PostgreSQL for anything and everything that needs to persist.
If postgresql is slow on certain queries that I cannot really make better than I either make a materialized view inside postgresql that updates on occasion, or if I do a ton of tiny little queries per transaction that all are the same (like getting permissions across connections, websockets, etc…) then I use CacheX (with a fallback in it set to querying the database) as it has auto-timeout and removal, janitor process, and far far more, including transactions and shared retrievals (it is built on ETS of course).

AstonJ

AstonJ

This is a good FAQ on Mnesia - any help?

http://erlang.org/faq/mnesia.html

11 Mnesia Database Questions

Mnesia is the somewhat odd name for the real-time, distributed database which comes with Erlang.

11.1 What is Mnesia good at?

Locking and Transactions

If you need to keep a database that will be used by multiple processes and/or nodes, using Mnesia means you don’t have to write your own access controls.

Distribution

Tables can be replicated at many nodes, both for efficiency (database lookup is a local activity) and robustness (redundancy means if one node goes down, other nodes still have copies of the data.)

Non-fully-normalised data

Unlike most database systems, records can contain data of arbitrary size and structure.

Monitoring

Monitoring - processes can subscribe to events which are sent when various operations on the data take place (update, delete, etc) The RDBMS package allows even more fine-grained control.

11.2 What is Mnesia not so good at?

Mnesia is primarily intended to be a memory-resident database. Some of its design tradeoffs reflect this.

Really large tables must be stored in a fragmented manner.

bentanweihao

bentanweihao

Author of The Little Elixir and OTP Guidebook

Then I think you can go with something like https://github.com/meh/amnesia. It’s querying syntax is pretty nice. But I would consider Ecto + Postgres an essential, well supported and documented dependency.

Where Next?

Popular in Questions Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New

Other popular topics Top

TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

We're in Beta

About us Mission Statement