NobbZ
Library to safely parse XML (by avoiding random atom creation)
I am writing an application where I need to parse XML, and intuitively I reached out for sweet_xml as its a wrapper around xmerl.
Though while I was inspecting the parsed result, I realized that attribute and element names appeared as atoms, and as the XML I receive will be beyond my control, I want to avoid “random” atom creation. Some HTML documents, that slip through could bring the system down…
So does anyone have a typ for an XML library I could use instead?
If it can directly transform the XML into a struct thats a plus, but not a strict requirement. Though if it can’t do structs directly, XPath is a requirement.
I explicitely do not want approaches that work like pythons xml2dict.
Most Liked
josevalim
voltone
There’s erlsom | Hex, which uses strings rather than atoms for element and attribute names…
voltone
Adzz
Just to follow up with what we ended doing at work after a lot of research.
We now use the default Saxy handler to spit out simple form, then we use data_schema’s to query the simple form. We implemented our own querying for simple form - which was not to difficult to do in the end. (If you are interested I may be able to open source).
This means you can cast the strings in the XML into data values quickly, then traverse the results using Elixir functions rather than xpath.
Not using xmerl and SweetXML drastically reduced our memory impact (we are talking from 1gb to 80mb for large XMLs) and sped it all up by about 12 times.
On top of that I’ve also implemented a custom Saxy handler that will only create the intermediate simple_form representation for values that are required by a data schema. So you pass a schema to the saxy handler then Saxy only creates simple_form for the values that the schema wants.
This keeps the memory impact very low as it never spikes higher than what is required by the schema. It adds a little complexity though.
deepankar-j
I ran into this issue of atom exhaustion vulnerability recently. In my case, this would have been a major issue because I was looking to parse SAML assertions received via a publicly accessible endpoint.
I ended up going a similar route as @Adzz and others by using saxy and its SimpleForm output.
To help avoid this repeated pain for others, I’ve created the following library, which avoids this atom exhaustion problem and includes the ability to verify XML signatures.
I welcome any community contributions or feedback
.







