This is definitely unexpected. I’m getting these results:

$ elixir -e "Mix.install [:benchee] ; Benchee.run([])" 2>/dev/null | head -7
Operating System: macOS
CPU Information: Apple M2
Number of Available Cores: 8
Available memory: 24 GB
Elixir 1.18.1
Erlang 27.2
JIT enabled: true
$ which elixir ; which erl
/Users/wojtek/.elixir-install/installs/elixir/1.18.1-otp-27/bin/elixir
/Users/wojtek/.elixir-install/installs/otp/27.2/bin/erl
$ hyperfine --warmup 2 -S none -- "elixir -e 'System.halt(0)'" "elixir -e 'IO.puts(:ok)'"
Benchmark 1: elixir -e 'System.halt(0)'
  Time (mean ± σ):     127.5 ms ±   1.1 ms    [User: 127.0 ms, System: 129.1 ms]
  Range (min … max):   125.7 ms … 129.6 ms    23 runs

Benchmark 2: elixir -e 'IO.puts(:ok)'
  Time (mean ± σ):     129.0 ms ±   0.9 ms    [User: 129.0 ms, System: 133.1 ms]
  Range (min … max):   126.9 ms … 131.0 ms    23 runs

Summary
  elixir -e 'System.halt(0)' ran
    1.01 ± 0.01 times faster than elixir -e 'IO.puts(:ok)'

Could you double-check you are benchmarking with elixir/otp installed via install script? What OTP are you using?

Most of time is spent booting the VM so I’d focus on that:

$ hyperfine --warmup 2 -S none -- "erl -noshell -eval 'halt()'"
Benchmark 1: erl -noshell -eval 'halt()'
  Time (mean ± σ):      90.7 ms ±   1.2 ms    [User: 92.4 ms, System: 89.5 ms]
  Range (min … max):    88.5 ms …  92.5 ms    33 runs

Perhaps it’s the OTP builds that the installer is using, GitHub - erlef/otp_builds: Community-maintained pre-compiled Erlang/OTP for macOS. What you can try is compiling OTP using kerl and re-running the benchmark. You can compile from scratch yourself too which is easy enough:

$ git clone https://github.com/erlang/otp --branch OTP-27.2 --depth 1
$ cd otp
$ ulimit -n 1024 && ERL_TOP=$PWD MAKEFLAGS=-j8 ./otp_build setup
$ ./bin/erl -noshell -eval 'halt()'
$ hyperfine --warmup 2 -S none -- "bin/erl -noshell -eval 'halt()'"

The ranges seem unusually large - benchmark 2 shows almost a 10x variation between the fastest and slowest run.

Is it possible your terminal is somehow running in Rosetta? That seems far less likely than it was in the early M1 days, but it’s the first thing I can think of that would produce that much inter-run variation (since it caches translated code).

One other random thing is what ulimits do you have set? I had to increase the laughable low default limit on my new M4 machine I set up recently.

More related to the “anyone noticed a drop in performance” link that you posted, and not really likely to be of any help, but a few years ago I noticed that elixir started up much more slowly than any of my other scripting languages. I had at one time I had a file showing the startup times of all of them but I can’t find it so I must have deleted it.

In any case, the problem went away. I can’t say that it went away because of an update to the OS or an update to elixir, and in fact I don’t remember ever knowing why it went away, but it did.

A colleague of mine, had performance issues, because the shell, he started all from, ran with Rosetta emulation. He switched from MPB with intell to one with M3 and did not do the full brew reinstall things.

Edit:
Sorry, coffee level is still low, so I did not see, that this idea already popped up