After some help and pointers from @josevalim I was able to make it work.

The updated repo is always here: https://github.com/xadhoom/sync_start_test and the summary is:

• create an umbrella containter
• create your primary app inside the apps dir, as usual. Add some start_phases to it (will the correct callbacks implemened)
• think about the apps that must be started as included_applications of the primary app. The primary app will become responsible of starting them
• keep in mind that mix will always start all apps inside the apps/ path
• create a new folder, like included_apps, at same level of apps/. Example: my_awesome_umbrella/included_apps
• put into included_apps your secondary app, with a subset of primary app start_phases (remind the callbacks)

Now the setup that makes it work

• in your primary app mix.exs change the mod value inside application/0 to mod: {:application_starter, [Primary.Application, []]}
• add an included_application config inside application/0, like included_applications: [:secondary]
• add a local path dep into deps/0 , like: {:secondary, path: "../../included_apps/secondary"}
• start your application, you will see start_phases callbacks called in correct order

As @josevalim pointed out, being :secondary out of apps path, the usual mix commands like mix test or other umbrella conscious commands will not run for apps outside the apps/ tree, so they must be run manually.

Caveats:
In order to create new apps inside the included_apps/ path, there’re two possible ways:

• create a new app using mix new inside the included_apps/ path and adjust all the needed project paths to point to umbrella:
• build_path: “…/…/_build”,
• config_path: “…/…/config/config.exs”,
• deps_path: “…/…/deps”,
• lockfile: “…/…/mix.lock”,
• or, more easily, create it under the apps/ path and then move it

Links:

• included_applications: http://erlang.org/doc/design_principles/included_applications.html
• synchronized start: http://erlang.org/doc/design_principles/included_applications.html#id82330