the_wildgoose

the_wildgoose

Elixir-reed_solomon_simd - wrappers for reed solomon encoding/ECC/FEC

Hi, I had a need for applying Reed Solomon protection to a (short) string. Think that you have a few hundred bytes and want to protect it against a couple of characters getting garbled, so you run it through the encoder, it creates a slightly longer string. Then later you can run (whatever makes it to the far end) through the decoder and repair any corruption. Think perhaps QR code protection, or protecting a long license key so it still works if you mistype, etc

However, a more common use case for Reed Solomon is to apply it to larger packets. For example PAR2 encoding slices a file into chunks and then encodes a protection stripped across all the chunks, so if a few were “erased”, you can recover them using the recovery packets

In a nutshell the encoding can be applied to even sized chunks and will protect against N/2 of those chunks going missing, where N is the amount of parity blocks you add

I wrapped two rust libraries. Why? Because I wrapped the high speed “reed-solomon-simd” library before realising that it wasn’t useful for my use case (protecting short strings), so I then wrapped a the simple reed-solomon-rs library which solves my use case. However, I’ve wrapped and released both libraries here:

Broadly you want the top one for protecting big lumps of data, such as IP packets, files, RAID on your disk, etc. And you want the second for encoding data on QR codes, or protecting small strings. Credit to the original authors as all I’m going is wrapping their work.

It’s my first attempt at writing some Rustler code, so feedback appreciated, also if anyone wants to give some guidance on how to pre-compile binaries for different platforms?

Most Liked

dimitarvp

dimitarvp

As I start I would recommend you not using .map_err to return a String. I’d make my own Error enum and have it implement Display (and maybe Debug as well) for such cases.

Or, even better, just implement rustler’s Encoder for it and map it to error tuples.

(Still use .map_err though, that’s how it’s done when you want to use ? and return Result-s but need your own custom error and want it serialized as an Elixir value.)

Or all of the above. I am doing that in my library (dimitarvp/xqlite on GitHub). Could send you a small PR if I have the time, though that’s not very likely, admittedly.

Where Next?

Popular in Libraries Top

josevalim
Hi everyone, We would like to announce that Plataformatec is working on a new MySQL driver called MyXQL. Our goal is to eventually integ...
New
mhanberg
I just released the first version of Temple: an HTML DSL for Elixir and Phoenix! You can read this blog post or the docs for more info...
New
tmbb
I’ve been working on two packages (not on hex.pm yet) to build admin interfaces for phoenix apps: bureaucrat - which contains a bunch ...
New
OvermindDL1
Been making an MLElixir thing (not released yet…) for fun in spare time in the past day. I’m just trying to see how much I can get an ML...
132 13487 106
New
archan937
It is a well-know topic within the Elixir community: “To mock or not to mock? :)” Every alchemist probably has his / her own opinion con...
New
Crowdhailer
I have been updating a library that allows you to pipe between functions that use the erlang result tuple convention. Assuming you have...
New
josevalim
Yes, yet another parser combinator library! Most of the parser combinators in the ecosystem are either compile-time, often using AST tra...
159 18262 141
New
wmnnd
Hi there, for my project DBLSQD, I needed a file storage solution that is a bit more flexible than Arc. Because I thought others might f...
New
OvermindDL1
I created a new library (rather I pulled out a couple files from my big project), it manages an operating system PID file for the BEAM. ...
New
Qqwy
While not as prevalent as in imperative languages, arrays (collections with efficient random element access) are still very useful in Eli...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

Sub Categories:

We're in Beta

About us Mission Statement