ahamez
Protox - A 100% conformant protobuf library
Hi everyone,
I’ve been working on this protobuf library for 3 years. We use it in the company I work for, EasyMile, to communicate with our autonomous vehicles.
As you can imagine, reliability is very important! Therefore, a lot of efforts have been put into making sure that the library is reliable. To this end, I committed to make it pass all the 1302 conformance tests that Google provides for binary serialization. This is on top of having a 100% code coverage, plus property based testing and dialyzer.
In order to assess the performance of protox, I benchmarked it with random and handmade inputs against the other available libraries, exprotobuf and protobuf-elixir. You can find these benchmarks here (zip archive of a benchee HTML export). They show that protox is always faster than exprotobuf and most of the time on par or faster than protobuf-elixir. Of course, the result might be different with another computer/OS and with different inputs, I’ll be happy to include new ones.
Also, I think that protox is easier to use as it let you work directly with Elixir structures (for instance, there’s no need to use a new function to create a protobuf message, which means that you’ll have a nice error if you use a wrong field name). Furthermore, there’s no need to generate any file.
But of course, I may be a little biased here 
.
[Edit] It seems that file generation is important for some users, so I guess I should find a way to make it possible after all!
You can find it here:
Let me know what you think!
Most Liked
ahamez
protox 1.0.0 has been released! As no new features have been added and the API has been stable for a long time, I thought it was finally the time to mark it stable.
So, as a recall: why would you want to use protox rather than the others libraries like protobuf-elixir which has more features (and users 
)? Well, It really depends on how much do you need a reliable implementation. As protox has been written for the case of autonomous vehicles, reliability was more important than features. Thus, it has been has been thoroughly tested using the conformance checker provided by Google, has a 100% test coverage and uses property based testing.
Don’t hesitate to give it a try 
.
ahamez
protox 1.2.1 is now available!
The main changes since 1.0.0 are:
- generation of one file per module, which speeds up the compilation if you have a lot of protobuf messages
- possibility to ignore unknown fields
And of course, it’s still 100% compliant with the Google conformance checker .
ahamez
protox 2.0 is out!
What it brings:
- encoding speed boost of around 50%
- significant internal restructuring for easier maintenance
- consistent way of accessing a message’s schema
Being a major release, there are breaking changes:
- drop support of JSON encoding/decoding (the implementation was half-baked and even Google doesn’t recommend using the JSON encoding). I hope people won’t be disappointed by this, but looking at the projects that use protox, none of them are relying on this functionality.
- encoding functions now return the size of the encoded data so you don’t need to compute it yourself (which, by the way, provided the encoding speedup)
- many generated functions to get a message schema information (like the syntax) are removed and the corresponding information are available in the structure returned by
schema/0.
ahamez
Even though Erlang 24 has been released 1 month ago, I thought it would be nice to assess the impact of Erlang 24 JIT on protox.
I ran some benchmarks yesterday, comparing with Erlang 23, and they show that it’s constantly faster with JIT:
- Decoding is 1.17-1.49 faster (average: 1.31), using 1.02-1.08 more memory (average: 1.06)
- Encoding is 1.10-1.92 faster (average: 1.58), using 1.03-1.17 more memory (average: 1.08)
A nice improvement in my opinion !
(You can perform theses benchmarks yourself using the instructions at https://github.com/ahamez/protox#benchmarks)
ahamez
protox 1.6 is out!
It’s a significant update as it provides support for JSON encoding and decoding, including Google well-known types (except Any).
Of course, Google conformance tests have validated that protox behaves as expected 
.
Popular in Libraries
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Enabling companies to succeed by building software people love.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"