NobbZ

NobbZ

Library to safely parse XML (by avoiding random atom creation)

I am writing an application where I need to parse XML, and intuitively I reached out for sweet_xml as its a wrapper around xmerl.

Though while I was inspecting the parsed result, I realized that attribute and element names appeared as atoms, and as the XML I receive will be beyond my control, I want to avoid “random” atom creation. Some HTML documents, that slip through could bring the system down…

So does anyone have a typ for an XML library I could use instead?

If it can directly transform the XML into a struct thats a plus, but not a strict requirement. Though if it can’t do structs directly, XPath is a requirement.

I explicitely do not want approaches that work like pythons xml2dict.

Most Liked

josevalim

josevalim

Creator of Elixir

I have also used saxy successfully. Specially useful if you want to get only certain parts of the XML.

17
Post #3
voltone

voltone

There’s erlsom | Hex, which uses strings rather than atoms for element and attribute names…

voltone

voltone

Adzz

Adzz

Just to follow up with what we ended doing at work after a lot of research.

We now use the default Saxy handler to spit out simple form, then we use data_schema’s to query the simple form. We implemented our own querying for simple form - which was not to difficult to do in the end. (If you are interested I may be able to open source).

This means you can cast the strings in the XML into data values quickly, then traverse the results using Elixir functions rather than xpath.

Not using xmerl and SweetXML drastically reduced our memory impact (we are talking from 1gb to 80mb for large XMLs) and sped it all up by about 12 times.

On top of that I’ve also implemented a custom Saxy handler that will only create the intermediate simple_form representation for values that are required by a data schema. So you pass a schema to the saxy handler then Saxy only creates simple_form for the values that the schema wants.

This keeps the memory impact very low as it never spikes higher than what is required by the schema. It adds a little complexity though.

deepankar-j

deepankar-j

I ran into this issue of atom exhaustion vulnerability recently. In my case, this would have been a major issue because I was looking to parse SAML assertions received via a publicly accessible endpoint.

I ended up going a similar route as @Adzz and others by using saxy and its SimpleForm output.

To help avoid this repeated pain for others, I’ve created the following library, which avoids this atom exhaustion problem and includes the ability to verify XML signatures.

I welcome any community contributions or feedback :slight_smile:.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
LegitStack
I’m hoping you guys can give me some general advice and perhaps code examples if you’re feeling up to it. I’m very interested in Elixir,...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 49522 488
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement