kingdomcoder
Parsing a Struct String - Code.eval_string/3? Custom Parser? Other Options?
After a few (successful?) attempts at creating DSLs to get structured data from users, I realized that Elixir structs’ syntax fits perfectly what we’re looking for.
My idea is to allow users to type in text that’s valid Elixir for given structs (atom keys) and then read/parse that into internal data structures.
I’ve considered using Code.eval_string/3, a thin layer on Poison’s custom decoders, and building a parser combinator from scratch with NimbleParsec.
How would you suggest I proceed (and why, please)?
These are the factors I’m evaluating to choose what approach to take
- The security of the approach
- The time and effort required to implement the approach
- The work required for future maintenance and extensibility
Any help is greatly appreciated.
Marked As Solved
hst337
Just use Code.string_to_quoted and traverse the AST. It is very easy to do
Also Liked
dorgan
Keep in mind that Code.string_to_quoted is as unsafe as String.to_atom. A new atom will be created for each key, identifier, etc in your input, so it’s an invitation to denial of service. You can avoid this by using :static_atoms_encoder but never, ever parse user input without taking that into account.
D4no0
Code.eval_string/3 is not safe, as you have full access to the runtime system, you can read the warning in the documentation.
kingdomcoder
I’ll try this and get back to you.
Thank you
kingdomcoder
You’re right. The AST is easy to parse.
I might settle for this approach.
Thanks for pointing this out.
kingdomcoder
Thanks, @kip
JSON would have been my goto approach.
However, there are two nuances I have to consider:
-
Any of
Parent,Child, andGrandChildin the example above can be the root node, and the system’s behavior adjusts accordingly. -
GrandChildcan be of multiple types and the keys for each are different (imagineTypeOne{some_key: "Value",...},TypeTwo{another_key: "Value"...}…)
If I go with JSON, handling these nuances might mean tagging each sub(object) with {"type": "NodeNameHere", ...}.
For our largely non-technical user base, requiring this tag (along with requiring quotes around each JSON key) will be too verbose.
Atom-keyed Elixir structs feel more succinct and expressive.
# JSON
{
"type": "Parent",
"field": "Value",
"another_field": "Another value",
"child_list": [
{
"type": "Child"
"child_field": "Child 1 data",
"grandchildren_list": [
{
"type": "TypeOne",
"field": "Value"
},
{
"type": "TypeTwo",
"another_field": "Some other value"
}
]
},
{
"type": "Child",
"child_field": "Child 2 data"
}
]
}
vs
# Struct
%Parent{
field: "Value",
another_field: "Another value",
child_list: [
%Child{
child_field: "Child 1 data",
grandchildren_list: [
%TypeOne{field: "Value"},
%TypeTwo{another_field: "Some other value"}
]
},
%Child{child_field: "Child 2 data"}
]
}
If there’s a workaround to reduce this verbosity, I’ll surely want to know 'cause parsing JSON will be an almost zero-stress approach for me.







