kingdomcoder

kingdomcoder

Parsing a Struct String - Code.eval_string/3? Custom Parser? Other Options?

After a few (successful?) attempts at creating DSLs to get structured data from users, I realized that Elixir structs’ syntax fits perfectly what we’re looking for.

My idea is to allow users to type in text that’s valid Elixir for given structs (atom keys) and then read/parse that into internal data structures.

I’ve considered using Code.eval_string/3, a thin layer on Poison’s custom decoders, and building a parser combinator from scratch with NimbleParsec.

How would you suggest I proceed (and why, please)?

These are the factors I’m evaluating to choose what approach to take

  • The security of the approach
  • The time and effort required to implement the approach
  • The work required for future maintenance and extensibility

Any help is greatly appreciated.

Marked As Solved

hst337

hst337

Just use Code.string_to_quoted and traverse the AST. It is very easy to do

Also Liked

dorgan

dorgan

Keep in mind that Code.string_to_quoted is as unsafe as String.to_atom. A new atom will be created for each key, identifier, etc in your input, so it’s an invitation to denial of service. You can avoid this by using :static_atoms_encoder but never, ever parse user input without taking that into account.

D4no0

D4no0

Code.eval_string/3 is not safe, as you have full access to the runtime system, you can read the warning in the documentation.

kingdomcoder

kingdomcoder

I’ll try this and get back to you.

Thank you

kingdomcoder

kingdomcoder

You’re right. The AST is easy to parse.

I might settle for this approach.

Thanks for pointing this out.

kingdomcoder

kingdomcoder

Thanks, @kip

JSON would have been my goto approach.

However, there are two nuances I have to consider:

  1. Any of Parent, Child, and GrandChild in the example above can be the root node, and the system’s behavior adjusts accordingly.

  2. GrandChild can be of multiple types and the keys for each are different (imagine TypeOne{some_key: "Value",...}, TypeTwo{another_key: "Value"...}…)

If I go with JSON, handling these nuances might mean tagging each sub(object) with {"type": "NodeNameHere", ...}.

For our largely non-technical user base, requiring this tag (along with requiring quotes around each JSON key) will be too verbose.

Atom-keyed Elixir structs feel more succinct and expressive.

# JSON
{
  "type": "Parent",
  "field": "Value",
  "another_field": "Another value",
  "child_list": [
      {
          "type": "Child"
          "child_field": "Child 1 data",
          "grandchildren_list": [
            {
              "type": "TypeOne",
              "field": "Value"
            },
            {
              "type": "TypeTwo",
              "another_field": "Some other value"
            }
        ]
      },
      {
          "type": "Child",
          "child_field": "Child 2 data"
      }
  ]
}

vs

# Struct
%Parent{
  field: "Value",
  another_field: "Another value",
  child_list: [
      %Child{
          child_field: "Child 1 data",
          grandchildren_list: [
            %TypeOne{field: "Value"},
            %TypeTwo{another_field: "Some other value"}
        ]
      },
      %Child{child_field: "Child 2 data"}
  ]
}

If there’s a workaround to reduce this verbosity, I’ll surely want to know 'cause parsing JSON will be an almost zero-stress approach for me.

Where Next?

Popular in Questions Top

gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New

Other popular topics Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement