wolf4earth
Babel - Data transformations made easy
Babel was born out of a desire to simplify non-trivial data transformation pipelines. To focus on the “happy path” instead of having to write a bunch of boilerplate error handling code.
But don’t listen to me, take a look for yourself:
pipeline =
Babel.begin()
|> Babel.fetch(["some", "nested", "path"])
|> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
data = %{
"some" => %{
"nested" => %{
"path" => [
%{"string-key" => :value2},
%{"string-key" => :value2},
%{"string-key" => :value2}
]
}
}
}
Babel.apply(pipeline, data)
=> {:ok, [
%{atom_key: :value1},
%{atom_key: :value2},
%{atom_key: :value3}
]}
Since you’ll most likely build non-trivial transformation pipelines with Babel - which can fail at any given step - Babel ships with elaborate error reporting:
Error Reporting
pipeline =
Babel.begin()
|> Babel.fetch(["some", "nested", "path"])
|> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
data = %{
"some" => %{
"nested" => %{
"path" => [
%{"unexpected-key" => :value1},
%{"unexpected-key" => :value2},
%{"unexpected-key" => :value3}
]
}
}
}
Babel.apply!(pipeline, data)
Which will produce the following error
** (Babel.Error) Failed to transform data: [not_found: "string-key", not_found: "string-key", not_found: "string-key"]
Root Cause(s):
1. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value1}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
2. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value2}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
3. Babel.Trace<ERROR>{
data = %{"unexpected-key" => :value3}
Babel.fetch("string-key")
|=> {:error, {:not_found, "string-key"}}
}
Full Trace:
Babel.Trace<ERROR>{
data = %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]}}}
Babel.Pipeline<>
|
| Babel.fetch(["some", "nested", "path"])
| |=< %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{...}, ...]}}}
| |=> [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
|
| Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
| |=< [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value1}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value1}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value2}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value2}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| | Babel.into(%{atom_key: Babel.fetch("string-key")})
| | |=< %{"unexpected-key" => :value3}
| | |
| | | Babel.fetch("string-key")
| | | |=< %{"unexpected-key" => :value3}
| | | |=> {:error, {:not_found, "string-key"}}
| | |
| | |=> {:error, [not_found: "string-key"]}
| |
| |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
|
|=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
}
Most Liked
wolf4earth
If you’re wondering whether or not Babel is production-ready: we’ve been using a pre-release version for nearly a year at this point at work in production, and it made external API integrations a lot easier and smoother. 
wolf4earth
What I’m taking away from this, is that more examples would be helpful and a comparison against libraries like Ecto.
kasvith
babel can be misinterpreted as the popular js transpiler
wolf4earth
I agree, the simple example I laid out can all be done easily with builtin functions.
But Babel isn’t meant to be used to simple data transformations. It’s meant to be used for non-trivial transformations. But putting that in a code example would make it a lot harder to grok and fail in it’s task to clarify the basic usage. Babel’s strength is in its composability, combining many simple elements into something that’s no longer simple but still understandable.
Doing these non-trivial transformations with standard library mechanisms is feasible but if you don’t want to raise and instead collect errors and be able to explain them, it becomes very complex - and also very hard to read - very quickly.
I’m planning to add a LiveBook which showcases how to use Babel to transform responses from GitHub’s GraphQL, which will lay out how the library shines for these non-trivial transformations.
wolf4earth
Just to give an example: here’s an excerpt from how we use Babel at work. I’ve changed model names and omitted some functions but the basic idea should be clear. The strength I’d argue is that it’s composable and readable in how it specifies the desired outcome, and if it fails at any given step it will tell you exactly what went wrong:
@spec list_of_pieces(opts) :: Babel.t([Piece.t()])
def list_of_pieces(opts) do
:pieces
|> Babel.begin()
|> Babel.fetch(["contentCollection", "items"])
|> Babel.map(piece(opts))
end
@spec piece(opts) :: Babel.t(Piece.t())
def piece(opts) do
Babel.into(%Piece{
id: cms_id(),
locale: locale(),
name: Babel.fetch("name"),
is_locked:
Babel.then(fn
%{"isPremium" => true} -> opts[:lock_premium?]
%{"isPremium" => _} -> false
end),
category: category(),
audio:
:audio
|> Babel.begin()
|> Babel.fetch(["audioCollection", "items"])
|> Babel.match(fn
[_ | _] -> Babel.at(0) |> Babel.chain(audio())
[] -> Babel.const(nil)
end)
})
end
This can then be used to parse like this:
Babel.apply(list_of_pieces(opts), data)
Or if you only want to parse a single Piece:
Babel.apply(piece(opts), data)
Popular in Libraries
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
The deployment platform built for Elixir: PaaS ease, VPS control.
Courses that'll move you from confusion to "Aha, now I get it!"