williamthome

williamthome

Exneus - JSON parser, generator and formatter for Elixir built on top of the new OTP json module

Exneus is an incredibly flexible and performant JSON parser, generator and formatter for Elixir. It is a wrapper for Elixir of Euneus, an Erlang library built on the top of the new OTP json module.

Both encoder and decoder fully conform to RFC 8259 and ECMA 404 standards and are tested using JSONTestSuite.

The new OTP json module was introduced in OTP-27, which is compatible with Elixir v1.17, but by installing json_polyfill, the minimum requirement reduces to OTP-24 and Elixir v1.12. I’ve not tested it in other versions, but the CI says it works.

Exneus/Euneus’s goal is to be very flexible and performant.

The first release of Exneus is an Erlangish lib. I just adapted Euneus’s Erlang code to Elixir. This will change :slight_smile:

Further explanation and examples are available at hexdocs.

Features

  • Encode
  • Decode
  • Stream
  • Minify
  • Format

Encode option details

  • codecs - Act like plugins. There are some built-in options, and it is also possible to create custom ones.

    Currently built-in encode codecs:

    • timestamp
    • datetime
    • ipv4
    • ipv6
    • records
  • codec_callback - - Overrides the default codec resolver.

  • nulls - Defines which values should be encoded as null.

  • skip_values - Defines which map values should be ignored. This option permits achieves the same behavior as Javascript, which ignores undefined values of objects.

  • key_to_binary - Overrides the default conversion of map keys to a string.

  • sort_keys - Defines if the object keys should be sorted.

  • keyword_lists - If true, converts keyword_lists into objects.

  • escape - Overrides the default string escaping.

  • encode_integer - Overrides the default integer encoder.

  • encode_float - Overrides the default float encoder.

  • encode_atom - Overrides the default atom encoder.

  • encode_list - Overrides the default list encoder.

  • encode_map - Overrides the default map encoder.

  • encode_tuple - Overrides the default tuple encoder.

  • encode_pid - Overrides the default pid encoder.

  • encode_port - Overrides the default port encoder.

  • encode_reference - Overrides the default reference encoder.

  • encode_term - Overrides the default encoder for unsupported terms, like functions.

Decode option details

  • codecs - Act like plugins. There are some built-in options, and it is also possible to create custom ones.

    Currently built-in decode codecs:

    • timestamp
    • datetime
    • ipv4
    • ipv6
    • pid
    • port
    • reference
  • null - Defines which term should be considered null.

  • binary_to_float - Overrides the default binary to float conversion.

  • binary_to_integer - Overrides the default binary to integer conversion.

  • array_start - Overrides the :json.array_start_fun/0 callback.

  • array_push - Overrides the :json.array_push_fun/0 callback.

  • array_finish - Overrides the :json.array_finish_fun/0 callback.

    In addition to the custom function, there are:

    • ordered
    • reversed
  • object_start - Overrides the :json.object_start_fun/0 callback.

  • object_keys - Transforms JSON objects key into Elixir term.

    In addition to the custom function, there are:

    • binary
    • copy
    • atom
    • existing_atom
  • object_push - Overrides the :json.object_push_fun/0` callback.

  • object_finish - Overrides the :json.object_finish_fun/0 callback.

    In addition to the custom function, there are:

    • map
    • keyword_list
    • reversed_keyword_list

Benchmark

The next step is to improve the performance since no optimization has been done yet. However, there is a simple benchmark comparing it with some known libraries, including the OTP json module.

IMPORTANT!

The benchmark result below just gives an idea of performance since the benchmark runs are very simplistic. Subsequent releases will update those values.

Also, to be an honest benchmark, all libs should produce the same output. I have not checked this.

Encode results:

Name                 ips
euneus           36.95 K
json (OTP)       35.85 K - 1.03x slower +0.83 μs
Exneus           35.71 K - 1.03x slower +0.94 μs
jiffy            34.70 K - 1.06x slower +1.76 μs
Jason            26.75 K - 1.38x slower +10.32 μs
thoas            16.40 K - 2.25x slower +33.92 μs
jsone            15.52 K - 2.38x slower +37.38 μs
jsx               4.21 K - 8.77x slower +210.29 μs
JSON              3.88 K - 9.52x slower +230.47 μs

Decode results:

Name                 ips
json (OTP)       18.79 K
euneus           18.06 K - 1.04x slower +2.16 μs
Exneus           17.68 K - 1.06x slower +3.34 μs
Jason            15.16 K - 1.24x slower +12.74 μs
jsone            13.82 K - 1.36x slower +19.17 μs
jiffy            12.76 K - 1.47x slower +25.15 μs
thoas            11.89 K - 1.58x slower +30.89 μs
jsx               4.93 K - 3.81x slower +149.64 μs
JSON              2.43 K - 7.74x slower +358.48 μs

If you liked those tools, find them helpful, and would like to see them improved, please consider sponsoring me or buying me some coffees :heart:

"Buy Me A Coffee"

Feel free to submit suggestions and to contribute :smiley:


Most Liked

williamthome

williamthome

Sorry, I wasn’t clear.

Using just the OTP :json module without any other lib is fine and is extremely fast! It provides encoding and decoding:

iex(1)> :json.encode(:foo)
[34, "foo", 34]
iex(2)> :json.decode("\"foo\"")
"foo"

However, the OTP :json module is available starting from OTP-27 and provides few options. On the other hand, :euneus is built on top of :json, providing more flexibility and options, including a JSON formatter.

For example, to decode Map keys as atoms:

iex(1)> :json.decode(~s({"foo":"bar"}), [], %{object_push: fn k, v, acc -> [{String.to_atom(k), v} | acc] end})
{%{foo: "bar"}, [], ""}
iex(2)> :euneus.decode(~s({"foo":"bar"}), %{object_keys: :atom})
%{foo: "bar"}

And that’s just one example. Also, I adapted the OTP json module into a lib called json_polyfill to increase the compatibility with OTP and Elixir versions, OTP-24 and Elixir v1.12, respectively.

Please let me know if this answers your question.

D4no0

D4no0

Quick question, why erlang devs need Euneus and can’t use the json library directly from OTP?

williamthome

williamthome

Thiago, I truly appreciate your perspective and respect your comment. Not only for Elixir, but Michał is also an extremely important person to the Erlang community, and his contributions are always great!

I do not know exactly what you mean by “fragmentation.”. Don’t get me wrong too, but I’m not inclined to start a discussion here comparing Exneus to another existing library. The main points of Exneus are in the thread.

Each library has its strengths and weaknesses. If you find a library that addresses your project’s requirements, feel free to consider using it. At the end of the day, libraries are tools, and choosing one over another is entirely optional. Moreover, libraries can always learn from and benefit from one another, and I would be delighted if Exneus could bring value to others. :blush:

williamthome

williamthome

There is no C library involved. The OTP json module is a pure Erlang module.

EDIT

The json_polyfill just adapts some parts of the code to be compatible with other OTP versions.

dimitarvp

dimitarvp

Yes it does, thank you.

Where Next?

Popular in Libraries Top

OvermindDL1
Been making an MLElixir thing (not released yet…) for fun in spare time in the past day. I’m just trying to see how much I can get an ML...
132 13487 106
New
Qqwy
While not as prevalent as in imperative languages, arrays (collections with efficient random element access) are still very useful in Eli...
New
versilov
Could not wait for the missing Elixir ML libraries to appear, so, I wrote one myself, taking https://github.com/sdwolfz/exlearn as a foun...
New
vic
Expat is a tiny experiment I did for extracting patterns and being able to reuse them (compose and share patterns between elixir librarie...
New
mplatts
With HEEX released we decided to start a components library using Tailwind CSS - check it out here: Petal Components. We also have a boi...
New
Crowdhailer
The latest release of Ace (0.10.0) includes serving content over HTTP/2. I have started writing a webserver to teach my self more about...
New
sasajuric
I’d like to announce a small library called boundaries. This is an experimental project which explores the idea of enforcing boundaries ...
New
treble37
Just looking for a little feedback on a tiny helper library I built - Sometimes I find the need to convert maps with atom keys to maps...
New
KallDrexx
For a good number of months I've been working on creating a very basic RTMP live video streaming server. Now that I have a very, very ba...
New
maltoe
Hello! Came here to announce ChromicPDF, a pet project PDF generator I’ve been working on for the past few months. Why another PDF gener...
New

Other popular topics Top

jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

Sub Categories:

We're in Beta

About us Mission Statement