vitaminwater
Umbrella project: applications inter communications, direct function call or message passing?
I’m working on a project that involves a phoenix frontend, an ecto backend, and scrappers.
The project is generated with phoenix 1.3 umbrella generator. So the frontend directly calls functions in the backend, and has the backend present in its deps, with in_umbrella option. Which I can understand as being a good solution as long as they are both on the same node.
Now I’m adding my scrappers, they should be able to interact with the backend, but it is not going to be on the same node. I need my scrappers to be able to move on different nodes, for example in the case when their IP address gets banned.
So if I understand right, I can’t add the backend in the scrappers’ deps, or it would start the backend app in the scrappers nodes. (Actually I’m not even sure that would be a problem…)
I wondering if I’m meant to create a GenServer in the backend that would act as some sort of a proxy for the nodes that can’t directly call its functions ? (and then add the backend in the deps, with the apps: false option)
Which also poses the question in the case where you’d like to have both phoenix frontend and backend on different nodes ?
Most Liked
vitaminwater
Thanks for the article Qqwy.
My question is more like: “If i’m forced to split my apps across nodes, what is the best to do in my case ?”
And my case is as follows:
I have 3 apps in an umbrella project.
- backend
- backend_web
- scrapper_ikea
I know that I should be able to split scrapper_ikea on separate nodes, mostly to have multiple instances in parallel with different IPs.
So now I’m wondering if I should have {:backend, in_umbrella: true} in the scrapper_ikea app ?
If understand correctly this would start the backend app too, which (as of now) is only my Ecto repo, doesn’t seem like too much of a problem to have multiple repos instances running on multiple nodes, right ?
Another option would be to have {:backend, in_umbrella: true, app: false} in the scrapper_ikea app ?
Then the backend app is not launched, but if I got it right means I’m not going to be able to interact with the database, and I’d only be able to call functions as long as they don’t rely on a particular process being present.
Which leads me to thinking, maybe I should have a GenServer launched with the backend application, that is only there to proxy calls to the repo, for use by the scrapper instances.
This way, I’d still be able to interact with the repo from the scrapper apps, but only through the GenServer API, which is kind of strange, as I’d have to depend on an app where most code is not usable… so maybe make a fourth app, only there to have this glue/proxy code ?
As you can see I’m quite deeply confused as to how to do all this.







