fireproofsocks

fireproofsocks

Best practices: Umbrella app with Multiple Repos

I’m hoping for some input from the wiser folks, but I’m wondering if there are some best practices when setting up an umbrella app with multiple repos. Freshly recovered from the sunburn of a legacy monolithic app, we have structured an umbrella app that seems to provide a good balance between the various concerns while maintaining an easily tested whole. It’s possible that each app under the umbrella uses its own database, but in practice, most of them will tap into the same PostGres instance.

However, one potential problem I’ve noticed came when we deployed the app: because many of the individual apps have their own repository, that means that the app requires more connections. We’re using Gigalixir and the basic first tier database supports 25 connections, but between 6 or 10 apps, that gets gobbled up pretty quickly, so for each app I had to whittle down the connections to 2 or 4 connections per app before the error messages went away.

My question for the group: is this a bad way to structure an umbrella app? I love the flexibility of being able to isolate services potentially into their own databases, but is it worth the extra headache of requiring more connections?

Thoughts welcome! Thanks!

Most Liked

LostKobrakai

LostKobrakai

Just to make this clear, there’s not a “single OTP app” in an umbrella. The top abstractions on the beam is an otp app and umbrella and poncho are both ways to compose multiple otp applications together. The difference is that an umbrella is a single mix project, which combines multiple nested “mix projects” into one – starting any of them by default, while poncho’s approach is developing each application in it’s own mix project and using simple {:dep, path: "…"} dependencies between each other, where there’s one “root” application, which depends on all the sub applications.

This sounds like you’re missing some abstraction here. For me MyApp.Repo is mostly a runtime construct handling db connections to a certain database (pooling and all that stuff). And it sounds like you want to reduce coupling to the app holding that module. Normally you’d do that by inversion of control. Create a behaviour in each app, which handles the concern of “accessing a db” and use configuration to let it use MyApp.Repo in prod and some other implementation in development/testing. If you want you could even reuse Ecto.Repo as the needed behaviour, as ecto already has a behaviour in front of the actual implementation within the library.

gregvaughn

gregvaughn

I agree that what you describe sounds more like poncho projects, which was pioneered by the Nerves team and also something Dave Thomas has used with his component-focused approach. The original writeup I’m familiar with is here https://embedded-elixir.com/post/2017-05-19-poncho-projects/

In my understanding I make a distinction between apps and projects. Projects are how code at rest (on the filesystem) are organized. Apps (in the OTP sense) are a runtime concern – basically a branch of a supervision tree. You can have a 1-to-1 mapping of project to app, or a 1-to-many. Umbrellas are projects with a 1-to-many mapping to apps. Things get more flexible when you also start thinking about releases. Releases are deployable units of apps. You can have multiple releases out of one umbrella, each using some subset of the contained apps. Distillery supports multiple release targets specifically for this purpose.

I like umbrellas. They fit well with the type of app I’m familiar with. I make one mitigating change from default umbrella setup to run each app’s tests in a separate BEAM to suss out any improper coupling early. Umbrellas may not fit everyone, but I’ve also heard a lot of misinformation.

jeremyjh

jeremyjh

This is where you lost me. If all these “Repos” are using the same Postgres instance, I’m skeptical there should be multiple Repos at all. Your apps can depend on each other, so you could have a shared Repo app that is used by all the different components, and yet each of those can have their own schemas, which may overlap or not depending on your architecture.

hauleth

hauleth

This isn’t true at all. Umbrella contain separate applications, and while sometimes these depends on each other it is not hard requirement and you can easily deploy them independently from each other.

Ponchos are just different form of keeping projects in the same repository. Nothing prevents you from using Poncho like Umbrella structure and vice versa (ok, using :in_umbrella will be harder, but that isn’t much of the problem).


You do not need Umbrella for that. Nothing prevents you from having multiple repos in the same application.

We cannot answer you to that question, as this highly depends. In your case you are very limited by the number of the connections, so you have your answer. In bigger systems? Who knows, sometimes it can be solution and sometimes it will not be.

BenMorganIO

BenMorganIO

This seems like a bad idea for umbrellas. Umbrellas are one app. They are. They are not one repo with multiple different apps. They are one app with multiple concerns. They are one app.

Do you want to feel deploy them as one or make seperate deployments? If that’s so, I think what you are looking at is poncho apps. Since they are their own report, I think you should move away from umbrellas and move into poncho.

As per the DB, this means memory usage will increase for each new app. If that’s ok, go for it. The amount of connections will be dependent on what each app requires. Normally I just worry about that later when load increases, but only if this current config can sustain the current load.

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

We're in Beta

About us Mission Statement