defp aliases do
  [
    …,
    "api_test": &api_test/1
  ]
end

defp api_test(_) do
  Mix.env(:test) # or whatever env you need
  MixTask.run("test" ["--only", "api_test:true"])
end

This is from memory though, can’t test it right now.

there’s also this in mix.exs:

def project do
    [
      …,
      preferred_cli_env: [
        coveralls: :test,
        "coveralls.detail": :test,
        "coveralls.post": :test,
        "coveralls.html": :test,
        "test.reset": :test,
        "test.integration": :test
      ]
    ]
  end

I know I am joining the party quite late, but I have seen a way in which you can actually execute an alias with a given ENV set, without having to manually type “MIX_ENV=blabla” before the command.

You can do it like they do in ecto, use a test_with_env function that does a lot of dark magic :

defp aliases do
    [
      "test.all": ["test.unit", "test.integration"],
      "test.unit": &run_unit_tests/1,
      "test.integration": &run_integration_tests/1
    ]
  end

  def run_integration_tests(args), do: test_with_env("integration", args)
  def run_unit_tests(args), do: test_with_env("test", args)

  def test_with_env(env, args) do
    args = if IO.ANSI.enabled?(), do: ["--color" | args], else: ["--no-color" | args]
    IO.puts("==> Running tests with `MIX_ENV=#{env}`")

    {_, res} =
      System.cmd("mix", ["test" | args],
        into: IO.binstream(:stdio, :line),
        env: [{"MIX_ENV", to_string(env)}]
      )

    if res > 0 do
      System.at_exit(fn _ -> exit({:shutdown, 1}) end)
    end
  end

You can read more about this in the blog:

Basically, the hardcore part is the command:

System.cmd("mix", ["test" | args],
        into: IO.binstream(:stdio, :line),
        env: [{"MIX_ENV", to_string(env)}]
      )

Which executes a command at system level and executes the given command with any variable you pass it, setting the ENV explicitly but without you having to care about it.

A clever use of aliases IMHO.

You are not allowed to call mix test in any other environment than :test, if you want to do it despite all warnings, then you have to set MIX_ENV explicitely. This is kind of saying “Yes, I know what I am doing and I am really sure I want to do this with all of its implcations and possible side effects which in worst case might destroy the world”.

If you really wan’t to use mix test in another environment, then there is no other way than to use the environment variable.

What’s the reason for using this “dark magic” over the simple method shown earlier?

Ecto uses such a complex method, because it tests multiple adapter setups, which are meant to work completely independant (postgres/mysql/mssql). If you’re not in a similar situation it can be much simpler.