gavid

gavid

Recursively replace all JSON `$ref`s with the code that they refer to

In JSON, it is possible in some formats (e.g. OpenAPI, React-JSON-Schema-Forms etc.) to use the $ref keyword as a rough parallel of macro functionality (see here: JSON References | NIEM GitHub).

This creates an issue when decoding JSON to Elixir (e.g. with Poison or Jason). I may be processing a complex JSON structure, and instead of the data I need being under "someKey", I have "$ref": "/path/to/something".

What I need is a way to replace all $refs with the actual JSON data that they represent. I have no idea how to do this, especially since $refs can be nested (i.e. a $ref refers to a JSON object with other refs inside it etc.). Also, in OpenApi, it is possible to add a attributes for summary and description after the $ref, but it is not possible to add any other attributes (How to use JSON references ($refs)).This creates further parsing complexity.

Is there some existing library that I can use maybe?

Marked As Solved

codeanpeace

codeanpeace

One approach would be preprocessing the JSON with a JS/TS library like json-schema-ref-parser to de-reference/resolve the references before decoding it in Elixir.

If you really wanted to do this in Elixir, I’d take a look at how the JS libraries handle tree traversal and re-implement it in Elixir. I imagine there would be opportunities for recursion.

Off the top of my head, Elixir’s built-in get_in/2 and update_in/3 functions in the Kernel module paired with the Access module will be helpful. There’s also a Pathex library that could be a good alternative for working with nested data structures.

Edit to add: this github issue might be relevant

Also Liked

al2o3cr

al2o3cr

Both of the examples you linked to are specifically about embedding references in schemas (either YAML- or JSON-formatted) to reduce duplication; I don’t believe a normal application decoding plain JSON would ever encounter a $ref outside of that context.

If you are expecting input like this, make sure to be careful about which URLs you follow and when. For instance, if you wrote a JSON parser that loaded these references before validating the shape of the data you could accidentally create a denial-of-service reflector. For instance, imagine a malicious payload like:

{
  "foo1": {
    "$ref": "http://example.com/oh_no1#foo"
  },
  "foo2": {
    "$ref": "http://example.com/oh_no2#foo"
  },
  "foo3": {
    "$ref": "http://example.com/oh_no3#foo"
  },
  "foo4": {
    "$ref": "http://example.com/oh_no4#foo"
  },
  ...etc for thousands of times
}

A single request to an API that tries to “follow” all those refs would be multiplied into thousands pointed at example.com.

Where Next?

Popular in Questions Top

openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New

We're in Beta

About us Mission Statement