Fl4m3Ph03n1x

Fl4m3Ph03n1x

How to have excoveralls consider total testing coverage from all apps in umbrella project?

Background

I am using excoveralls in an umbrella project with several children.

I have set up the children to use Excoveralls like this:

For normal Elixir applications:

 def project do
    [
      app: :app1,
      version: "3.1.0",
      build_path: "../../_build",
      config_path: "../../config/config.exs",
      deps_path: "../../deps",
      lockfile: "../../mix.lock",
      elixir: "~> 1.16",
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      test_coverage: [tool: ExCoveralls],
      preferred_cli_env: preferred_cli_env()
    ]
  end

And for a Phoenix child, the configuration is slightly different:

def project do
    [
      app: :phoenix_app_1,
      version: "2.2.1",
      build_path: "../../_build",
      config_path: "../../config/config.exs",
      deps_path: "../../deps",
      lockfile: "../../mix.lock",
      elixir: "~> 1.16",
      elixirc_paths: elixirc_paths(Mix.env()),
      compilers: Mix.compilers(),
      start_permanent: Mix.env() == :prod,
      deps: deps(),
      test_coverage: [tool: ExCoveralls],
      preferred_cli_env: preferred_cli_env()
    ]
  end

 defp deps do
    [
      # Phoenix
      # ....

      # I dont know why I have to explicity include excoveralls here
      # but if I dont `mix coveralls` wont work for Phoenix child apps
      {:excoveralls, "~> 0.18", only: :test},
    ]
  end

For both types of apps the preferred_cli_env set to this:

defp preferred_cli_env,
    do: [
      coveralls: :test,
      "coveralls.detail": :test,
      "coveralls.post": :test,
      "coveralls.html": :test,
      "coveralls.github": :test
    ]

Problem

When I run mix coveralls it runs all tests from each child app sequentially, due to my customized mix test alias:

defp aliases do
    child_tests =
      Path.wildcard("apps/*")
      |> Enum.map(&String.replace(&1, "apps/", ""))
      |> Enum.map(fn app -> "cmd --app #{app} mix test --color" end)

    [test: child_tests]
  end

So what ends up happening, is that coveralls will run the coverage for each app, and then only present me with the coverage for the last app run.

This is a problem. I don’t want the coverage of only the last child app, I want the coverage of all child apps, inside the project, so i know the overall test coverage of the project.

Questions

  • Is this possible to do with coveralls?
  • If so, how?

Marked As Solved

LostKobrakai

LostKobrakai

Named processes are a common pattern for sure. The issue is also not the named process by itself. The issue is the hardcoded dependencies on those names (and their backing process) into your codebase, which means tests can no longer opt out of using that named process in favor of another instance.

As soon as you look into applications we would generally consider to be libraries you see the pattern of keying a whole tree of processes to a given name all the time. E.g. in ecto processes are keyed to the repo name, in broadway the processes are keyed to the name of the broadway pipeline. E.g. see here how the MyBroadway name determines the names of all child processes: Broadway — Broadway v1.0.7

Also Liked

LostKobrakai

LostKobrakai

Tbh that sounds like you’re manually starting applications, which you generally shouldn’t need to be doing in the first place. If you have your dependencies setup correctly between your umbrella applications (using {:app, in_umbrella: true}) all necessary applications should start up no matter what mix task you run and if it’s for the whole umbrella or just a single app within.

LostKobrakai

LostKobrakai

Fair. You got global state in your application and your tests show you that this is not necessarily the best design.

My general suggestion for global state is refactoring those singleton processes (with a fixed name) to processes, which can be given a name. That way you can start multiple concurrent instances of those processes with different names.

Given that you can start a separate set of processes per test. You can use dependency injection (of various forms) to inject the name (or names) to be used by your business logic when interacting with those processes.

Production can use a single name (or names), which effectively gives you singleton behaviour in production.

The issues you run into are why I generally suggest that test should either not depend on state present on processes started by applications or at least that state should be keyed to the running test. That approach is essentially how the ecto sandbox or mox allow you to run tests concurrently.

While this is talking about processes the same can equally be applied to other global resources.

LostKobrakai

LostKobrakai

Usually, but I tend to mirror the order of parameters GenServer.call has as well. It’s imo the more reasonable order if you happen to pass both the server and item.

Where Next?

Popular in Questions Top

logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

Other popular topics Top

chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New

We're in Beta

About us Mission Statement