gavid

gavid

Recursively replace all JSON `$ref`s with the code that they refer to

In JSON, it is possible in some formats (e.g. OpenAPI, React-JSON-Schema-Forms etc.) to use the $ref keyword as a rough parallel of macro functionality (see here: JSON References | NIEM GitHub).

This creates an issue when decoding JSON to Elixir (e.g. with Poison or Jason). I may be processing a complex JSON structure, and instead of the data I need being under "someKey", I have "$ref": "/path/to/something".

What I need is a way to replace all $refs with the actual JSON data that they represent. I have no idea how to do this, especially since $refs can be nested (i.e. a $ref refers to a JSON object with other refs inside it etc.). Also, in OpenApi, it is possible to add a attributes for summary and description after the $ref, but it is not possible to add any other attributes (How to use JSON references ($refs)).This creates further parsing complexity.

Is there some existing library that I can use maybe?

Marked As Solved

codeanpeace

codeanpeace

One approach would be preprocessing the JSON with a JS/TS library like json-schema-ref-parser to de-reference/resolve the references before decoding it in Elixir.

If you really wanted to do this in Elixir, I’d take a look at how the JS libraries handle tree traversal and re-implement it in Elixir. I imagine there would be opportunities for recursion.

Off the top of my head, Elixir’s built-in get_in/2 and update_in/3 functions in the Kernel module paired with the Access module will be helpful. There’s also a Pathex library that could be a good alternative for working with nested data structures.

Edit to add: this github issue might be relevant

Also Liked

al2o3cr

al2o3cr

Both of the examples you linked to are specifically about embedding references in schemas (either YAML- or JSON-formatted) to reduce duplication; I don’t believe a normal application decoding plain JSON would ever encounter a $ref outside of that context.

If you are expecting input like this, make sure to be careful about which URLs you follow and when. For instance, if you wrote a JSON parser that loaded these references before validating the shape of the data you could accidentally create a denial-of-service reflector. For instance, imagine a malicious payload like:

{
  "foo1": {
    "$ref": "http://example.com/oh_no1#foo"
  },
  "foo2": {
    "$ref": "http://example.com/oh_no2#foo"
  },
  "foo3": {
    "$ref": "http://example.com/oh_no3#foo"
  },
  "foo4": {
    "$ref": "http://example.com/oh_no4#foo"
  },
  ...etc for thousands of times
}

A single request to an API that tries to “follow” all those refs would be multiplied into thousands pointed at example.com.

Where Next?

Popular in Questions Top

fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
clayschick
I'm trying to create a simple query to select distinct values from a column. If I only use select and distinct I get back a list of uniqu...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

We're in Beta

About us Mission Statement