krisleech

krisleech

Deserialized Avro, using AvroEx, contains incorrect data

When deserializing Avro, using AvroEx, I was getting an error, so I decided to start with a simple example to see how thing work.

However while I don’t get an error in this case I do get the wrong data back.

The data is serialized in a Ruby application as follows:

id = 1

encoder = AvroTurf.new(schemas_path: Rails.root.join("kafka_schemas"))
message = encoder.encode({ "a" => id}, schema_name: "test")
File.write("/tmp/message.avro", message)

and then deserialized as follows in Elixir:

schema = AvroEx.decode_schema!(File.read!("#{:code.priv_dir(:kafka)}/schemas/test.avsc"))
p = File.read!("/tmp/message.avro")
%{ "a" => id } = AvroEx.decode!(schema, p)

id # => -40

As you can see I get a different id back, -40 versus 1.

The schema both are reading is as follows:

{
  "type": "record",
    "name": "test",
    "doc": "Test",
    "fields": [
    {
      "name": "a",
      "type": "int"
    }
  ]
}

Is it possible this is due to the encoding of the file, versus raw bytes… I tried using File.writebin on the Ruby side and it creates the same file (same checksum).

Many thanks!

Marked As Solved

krisleech

krisleech

I tried using Avrora and it works just fine. I’m not sure but I think the issue might have been because AvroEx works with plain Avro without headers or embedded schema.

Where Next?

Popular in Questions Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
273 38985 115
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement