hansonkd
Dynamic function generation at runtime with Code.eval_... very slow
Hi.
I am making a library for user submitted templates (so users can customize email content). I have a parser which parses the template to an AST, and now I am implementing evaluating the template. My first pass was to interpret the template, going over the AST every time it renders, then my second pass was to build an anonymous function from the AST (so it is recursive anonymous functions for the different branches of the AST), now on my 3rd pass, I am building up an Elixir AST which in turn evaluates to a function.
My code uses quoted expressions, but it is easier to show the problem using strings:
with this string:
{compiledB, _} =
Code.eval_string("""
fn vars ->
%{"my_var" => userVar_0} = vars
[["(", to_string(userVar_0), ":", to_string(userVar_0), ")"]]
end
""")
Benchmark.measure(fn ->
%{"my_var" => 20}
|> compiledB.()
|> IO.iodata_to_binary
end)
Where Benchmark.measure calls the function 1000 times takes 0.025493 seconds, which is slower then interpreting it.
If I change it to do this:
Code.compile_string("""
defmodule A do
def render(vars) do
%{"my_var" => userVar_0} = vars
[["(", to_string(userVar_0), ":", to_string(userVar_0), ")"]]
end
end
""")
Benchmark.measure(fn ->
%{"my_var" => 20}
|> A.render()
|> IO.iodata_to_binary
end)
Running this 1000 times takes 4.37e-4 seconds
I like eval_string, since it returns an anonymous function right away and the function is isolated from the rest of the execution environment. Using the compile option, I can’t compile it into an anonymous function, only a module, which then gets put into the Elixir environment so I would have to make sure Module names are unique (but not too unique so I don’t run out of atoms). I would also need to purge manually any templates that aren’t used anymore, whereas with eval_string, I assume the garbage collector would handle cleaning them up automatically.
Why is eval_string several orders of magnitude slower? Is using compile_string and then keeping track of module names and purging them when no longer needed the way to go?
Thanks!
Most Liked
michalmuskala
Only code inside modules is compiled, code outside modules is always interpreted through erl_eval. So even when you define the fun inside Code.eval_string you’re not compiling the function - what you receive is a reference to a function that will be interpreted when executed. That’s why threre’s not much performance difference and why code inside a module is so much faster.
josevalim
The purpose of the program definitely matters, because there may be simpler solutions available.
When you are evaluating code at runtime, there are at a minimum 4 or 5 compiler passes involved, to build something that is then evaluated (i.e. invoked by hand and not compiled). In your case, it is not clear why you need to use eval in the first place. Couldn’t you return the function?
compiledB = fn vars ->
%{"my_var" => userVar_0} = vars
[["(", to_string(userVar_0), ":", to_string(userVar_0), ")"]]
end
hansonkd
Yes, you are probably right that interpreted is probably enough since IO to send the email will most likely be the limiting factor, performance wise.
Just incase anybody reads this in the future, my results on a small template with nested loops.(“compiledFully” uses Code.compile outlined above, “semi-compiled” is traversing the AST and building nested anon functions).
Compiling once then calling render (compile is outside loop)
Name ips average deviation median 99th %
compiledFully 32.78 M 0.0305 μs ±23.34% 0.0290 μs 0.0580 μs
semi-compiled 0.0337 M 29.71 μs ±33.99% 27 μs 79 μs
Comparison:
compiledFully 32.78 M
semi-compiled 0.0337 M - 973.78x slower
Compiling every render (compile is inside loop)
Name ips average deviation median 99th %
semi-compiled 9.83 K 0.102 ms ±342.27% 0.0920 ms 0.180 ms
compiledFully 0.105 K 9.50 ms ±3.93% 9.49 ms 10.42 ms
Comparison:
semi-compiled 9.83 K
compiledFully 0.105 K - 93.37x slower
So if you expect to use the compiled template more then 10-50x, it is worth it to compile it to Elixir.
josevalim
Right, the compiled approach will always be faster. If you have a general template engine, then the compiled approach will be faster and probably simpler.
However, if your template engine is limited (think mustache/handlebars), then I would precompile only the template into a Template AST and not necessarily compile it down to Elixir. Evaluating this template ast should be reasonably fast as long as you rely on IO lists (i.e. never concatenate strings, only keep them in a list).
I am not sure if this helps but these would be my two cents. I would avoid eval generally though.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
The deployment platform built for Elixir: PaaS ease, VPS control.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"