jeroenhouben

jeroenhouben

Jason decode string and keys options, what do they do?

I’m reading the options in Jason.decode but the docs don’t explain it well enough for me to fully understand the pros and cons of the options:

https://hexdocs.pm/jason/Jason.html#functions

I’m parsing a long list of JSON files into Structs and storing those in a long running Agent. Which options should I be using?

TIA,

Jeroen

Marked As Solved

lucaong

lucaong

Then :copy might be better, especially if it is fast enough for your case, but you could try to check the memory usage: in most cases it won’t make much difference. The worst case scenario is when you have a huge JSON, but you only save for long time a tiny part of the decoded data: the whole big JSON binary would stay in memory, even if you only use a small part of it.

Also Liked

lucaong

lucaong

The keys option controls whether keys are decided into strings or atoms. Atoms are never garbage-collected, so if the JSON can contain arbitrary keys, you should decode keys as strings to avoid the possibility that a malicious user would send many different keys, creating many atoms and causing your app to leak memory. If you expect the keys to belong to a known set of possible values, you could use :atoms!, which would decode keys to atoms but only allow existing atoms, not create new ones, to prevent the case above.

The strings option controls performance optimization in string keys and values. The end result is the same, but there are performance considerations to take into account. The default is to reference string keys and values: the decoded data would reference the original binary when possible, instead of copying over the strings. In other words, if the original JSON string was "{\"foo\": 123}", and the decoded data is %{"foo" => 123}, the "foo" key in the decoded data would not be a new string, but “point” to the part of the original JSON string that contains foo. This will save memory and time, as no copies of the strings need to be created, but could create memory leaks if you store pieces of the decoded data in memory for a long time: since the decoded data might be referencing the original binary data, the latter cannot be garbage collected. If you don’t store the parsed data in memory, but rather just use it, you can safely use the default here.

Is this clearer?

lpil

lpil

Creator of Gleam

To add to the excellent info from @lucaong, if you don’t know specifically what you want then the defaults are best :slight_smile:

jeroenhouben

jeroenhouben

Well first of all many thanks for this elaborate answer!

If you don’t store the parsed data in memory, but rather just use it, you can safely use the default here.

I read the json from files and then I create structs from the parsed data. And I store those structs in memory… I think I should use :copy to avoid mem leaks then… ?

Where Next?

Popular in Questions Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
Kagamiiiii
Student & New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
gazoon
I want to know absolute current module path. In python i could do that: os.path.abspath(__file__) Does elixir have anything similar?
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

Other popular topics Top

josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New

We're in Beta

About us Mission Statement