Menkir
Jexon - Lossless encoding to JSON
Hi everyone,
I recently released a small library built on top of Jason that converts Elixir maps and structs to JSON without loss. Atoms and tuples are data types that have no direct representation in JSON. As a result, keys and values of these data types are specially encoded with prefixes. Additionally, structs without the @derive attribute can be directly encoded to JSON.
The library offers three functions:
to_map: This function formats structs into maps and stores the__struct__key.to_jsonfrom_json
Feel free to contribute in case you have more edge cases to consider.
Happy Coding!
Most Liked
adamu
Sorry to be that guy, but if the purpose is to have kind of transport encoding with the objective of eventual decoding back into Elixir, it seems that external term format would be a good choice, unless you need JSON for other interoperability reasons. Maybe some suggested use cases or a mention of this in the docs would be useful ![]()
Menkir
I fully understand this. We initially considered this formatting, but we had several concerns:
If you want to transport the state via JSON (for instance, for persistence), it’s necessary to encode it into Base64. Subsequently, you can store it in any DB of your choice. Yet, when you try to retrieve this state, you’re confronted with binary data, which is incomprehensible to both the remote machine and to you.
With Jexon-encoded Elixir structs, it becomes feasible to perform diffs between varied states or carry out migrations when certain keys have changed.
Debugging is more straightforward than when using ETF. I can effortlessly copy and paste parts or the entire state into a remote IEX, then pipe it through functions to diagnose the issue.
Ultimately, the decision boils down to preference: Would you rather store your data in binary or JSON format? In my specific situation, the data was initially stored locally in ETS. However, as is often the case, requirements shifted, necessitating the state’s transfer to a centralized backup service that exclusively accepts JSON.
mayel
Looks interesting! Some examples of the JSON output would be useful in the readme.
al2o3cr
One tricky extreme edge-case: Jexon.to_json will produce the same output for these two maps
%{"__atom__:foo" => 1}
# vs
%{foo: 1}
A less edge-case situation: what about maps with other kinds of terms as keys? For instance, a lot of “sparse grid” problems make sense as a map of {x, y} tuples to values:
game_of_life_grid = %{
{0, 0} => 1,
{0, 1} => 2,
# etc
}
Menkir
For the extreme edge case: Yes, both lead to the same result because atoms would be encoded as strings in any other Encoder like Jason, Poison etc… If you want to keep the information about the type so that you can parse it back to the original data structure you need to encode the type like __atom__:... otherwise you will never know if it’s a string key or an atom. You can try it oppisite: Take the JSON output and parse it back you will always get
%{foo: 1}`
If you take Jason or Poison you need to pass an option like keys: :atoms so it ends up to the developer and not to the data itself which types it has.
Tuple keys are a good one. I didn’t consider tuples as keys. Will fix it soon. Thanks!








