In Membrane, we use nested structs and (usually small) maps for element state, which is basically a GenServer state. There are rather no heavy loops over it, but we figured out that access to structs impacts performance significantly. So we replaced Elixir’s Access by macros that construct pattern matches in the compile time. That turned out to provide significant speedup in some of our benchmarks. Not sure if that’s what you’re looking for, but I guess speeding up structs/maps themselves could give Membrane another boost.

It’s been some time, but the benchmark we had wasn’t straightforward to run, so I made another one: membrane_benchmarks/core_beamchmark at master · membraneframework-labs/membrane_benchmarks · GitHub

Hope it’s useful

Here’s some prior discussion that seems like a good starting point:

A big change worth learning about for this specific case (Elixir structs) is the new_map_lit machinery that landed in OTP 21:

I think rebuilding Commanded’s aggregate from a lot of events (when there’s no snapshot) could be a good example of that.
We’re doing that in our company too. I can provide you with some test code and data or maybe even a Benchee script to run, if you want.

Oh!

Here’s a thing that might help. I’ve been (experimenting) with XML parsing. One iteration I did was creating a Saxy handler that parses the XML into a struct, something like the following,

Given this xml:

<ParentNode price="1">
  <MyNode>Stuff</MyNode>
  <AnotherNode>
    <Child>Stuff</Child>
  </AnotherNode>
</ParentNode>

The Saxy handler returns:

 %XMLNode{
   attributes: [%XMLAttr{name: "price", value: "1"}],
   content: [
     %XMLNode{attributes: [], content: ["Stuff"], name: "MyNode"},
     %XMLNode{
       attributes: [],
       content: [%XMLNode{attributes: [], content: ["Stuff"], name: "Child"}],
       name: "AnotherNode"
     }
   ],
   name: "ParentNode"
 }

THEN we have a library called data_schema that queries into that representation to create a different struct from it. It does that by reducing through a schema and querying into the result from Saxy, so if you have large XML you’ll be doing lots and lots of Map.gets into the result of the Saxy handler to get the values for your new struct.

I have created a branch on the data_schema repo called xpath_experiment that has the Saxy handler and a data accessor that you can use to parse XML.
See this example from the tests:

Doing this will result in a lot of Map.fetch into a struct - 1 per node and attr.

If you think this will be useful to you I can get together a large XML (~15mb) example to use, along with schemas for that.