jstimps

jstimps

Writing a new Ecto Adapter (not SQL), unable to use Ecto Migrations effectively

Hi,

I’m working on an Ecto Adapter for FoundationDB and one of my goals is to support migrations in the same manner that Ecto users are accustomed to (e.g. mix ecto.migrate). As I familiarized myself with the adapter behaviours, I learned that the Ecto.Adapter.Migration behaviour is a module in ecto_sql instead of the base ecto.

My first question: Is it reasonable or folly to continue developing a migration using ecto_sql as a depedency even though my database backend does not support SQL?

Of course I’m no stranger to folly, and I went ahead and tried it anyway, and got it mostly working. The rest of this post is related to the one blocker I had in getting it working seamlessly.

My desired implementation for Ecto.Adapter.Migration.lock_for_migrations/3 acquires a resource (a transaction reference from :erlfdb) that must then be telegraphed down to each execute_ddl. Ideally this could be directly supported by ecto_sql by allowing the behaviour to influence any future Repo operations within the execution of the migration.

Without this direct support from the library, a quick and dirty approach would be to store the reference in the Process dictionary, and then pick it back up later on in execute_ddl. However, this doesn’t work either because of the following Task creation:

Ecto.Migrator

    fn -> run_maybe_in_transaction(repo, dynamic_repo, module, fun_with_status, opts) end
    |> Task.async()
    |> Task.await(:infinity)

(The use of a Task here ensures that the rest of the migration steps have a fresh Process dictionary, preventing me from using it to keep track of data)

My second question: Why does ecto_sql use a Task in this manner?

Finally: Are there any other solutions that I’m missing without changing the underlying ecto_sql implementation? I’m reluctant to use global state such as an ets table unless it’s a last resort.

Thanks for your time!

Jesse

Most Liked

Schultzer

Schultzer

Since the adapter is unaware of the user setup, global is a good choice, as it works regardless of whether the node is in a cluster. The adapter is still very much in the exploration stage but is usable. With the occasional bugs here and there that I’m finding while building a different product.

I think it might be worth proposing in https://groups.google.com/g/elixir-ecto to make Ecto.Migrator a behaviour and move that and Ecto.Migration into ecto main and have adapters implementing them since we both could use the flexibility and control that would bring. And it might get some more clarity over the choice already made, which is unclear for both of us, and I’m sure there will be other people in the future running into this.

/edit
Keep in mind that some people don’t use global since AFAIK the regular Erlang distribution is a full mesh, and that can bring some pain when running a large cluster, However, I don’t believe that running one large cluster is a silver bullet, and there might be more gains by divide and conquer and running multiply full mesh clusters that can communicate.

Where Next?

Popular in Questions Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement