abitdodgy

abitdodgy

Different apps, shared database, different schema... a good idea?

We would like to use a shared database for a number of separate applications. These apps are completely separate (separate repo, deploys, etc…). Many of these apps have interdependencies between their data. For example, an application for creating and sharing project estimates would need access to product data in a catalog app that furnishes an API. Also, all of these apps would need authentication, which would be handled by an identity app that keeps user data.

There are several approaches we can take:

  1. Integrate the apps over APIs. I don’t like this idea because it might mean recreating tables and schemas in two separate databases and then worrying about keeping data in sync. For example, we might need to create a user and product tables and their schemas in both apps. Although, an API serves as a contract and could make the integrations more robust.
  2. Keep all apps in an umbrella or monolith. I don’t like this option because these apps deal with different business concerns, and while they might share underlying data, they implement this data in completely different ways.
  3. Separate apps, shared database. I don’t like this because, unless you are mad, you would need a dedicated app to manage your repo and migrations.
  4. Separate apps, shared database, separate database schemas. This is my preferred option.

I prefer option four because it’s easier to work on and maintain separate applications. Separate schemas ensure that each app can manage its own database. But a shared database makes it easier reuse data, setup joins, and database constraints or maintain referential integrity. It’s as easy as joining a different prefix.

For example, an estimates.cart (schema.table) could have an owner_id foreign key that points to identity.user. And estimates.line_items can have a product_id column that could point to catalog.products.

Does this sound like a good idea? Has anyone done this type of architecture? What disadvantages (other than some schema data being shared between migrations) do you see?

Would you go down this route, or would you recreate data where needed? For example, multiple user schemas and so on…

Most Liked

anthonator

anthonator

We’ve done a combination of 2 and 4.

We have an umbrella app that uses a shared database and the data layer of each app is separated by Postgres schemas.

Our approach used the onion architecture. We have 3 different layers within our application.

  1. infrastructure - This layer is where different infrastructural applications live. Our repo lives in this layer. An event bus. Etc.
  2. context - We keep all of our business logic in this layer. These applications expose their functionality through a public interfaces that encapsulate the business processes for each application.
  3. service - These applications are how the outside world interacts with our application. This is where our REST and GraphQL API’s live.

The dependency flow within our application looks like this.

[service] —> [context] —> [infrastructure]

This means that our service applications can directly interact with our context and infrastructure applications but they cannot directly interact with other service applications. Context applications can only directly interact with infrastructure applications and infrastructure applications don’t directly interact with anything. If information needs to be be passed to applications in the same layer or the layer above then we dispatch events using a bus. Applications in the same layer or the layer above will never depend on each other. This helps make the dependency chain and information flow between the various applications easier to reason about.

The context layer is where all the database querying and mutating happens. Each context application’s data is “isolated” from the other. I put isolated in quotes because they are separated by Postgres schemas. So it’s possible to query between the schemas but we have strict rules preventing that. Each application is responsible for it’s own data. Hard stop. If some piece of logic needs information that typically comes from another context application’s database then it’s queried for in a service application and passed in as an argument to a function. We essentially pretend that each Postgres schema is a separate physical database.

We’ve been using this approach for about 2 years and it’s worked very well for us. It’s made it easier to manage the application as it’s grown. It also promotes loose coupling which makes the application easier to change as well.

al2o3cr

al2o3cr

These two statements are in direct conflict: if you’re pulling records from a schema that belongs to another “separate app” you’ve created a coupling between the code that reads that data and the code that writes it. For instance, renaming a column in identity will have to be handled with coordinated deploys of every caller that uses the renamed field.

IMO this is a good use case for an umbrella app containing multiple Ecto.Repo applications.

hauleth

hauleth

What do you mean by the “shared database”? Is this “shared DB installation” or “shared DB within Postgres with separate Postgres Schemas”? If first one then I would say that this is almost no different from different DB instances on different machines.

LostKobrakai

LostKobrakai

In an umbrella project you can build releases with just a subset of applications included. So you can have all your different projects and their shared dependencies in an umbrella and still have their releases only include the parts they actually need.

anthonator

anthonator

The repo is shared between all applications so the prefix is declared in each schema.

We share the repo so we can use the same connection pool between each application.

Where Next?

Popular in Questions Top

vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New

We're in Beta

About us Mission Statement