Phillipp

Phillipp

Help cleaning up this data transformation code

Hey,

I got a list of structs which I need to convert to a map (or similar useful data structure) that I can then further use to create some Prometheus metrics.

Here is what I got. The code is quite dirty in my opinion and maybe there is a smarter way to do it. I wrote it last night at 1am just to get it done, don’t judge me :smiley:

Input data:

trackers = [
  %{
    pause: true,
    pause_timeout_in_ms: 72276156,
    position: %{"lat" => 11, "lng" => 1},
    response: nil,
    timeout_in_ms: 309722,
    user_id: "5cb6fb2c2071c963d21c517f"
  },
  %{
    pause: false,
    pause_timeout_in_ms: false,
    position: %{"lat" => 11, "lng" => 1},
    response: %{
      "status" => "ok",
      "vehicle_position" => %{"vehicle" => %{"tracking" => nil}}
    },
    timeout_in_ms: 1747683,
    user_id: "5cb6fb2d2071c963d21c5180"
  }
]

Then I do the following:

trackers
    |> Enum.map(
         fn x ->
           state = case x[:pause] do
             true -> "paused"
             false -> "active"
           end
           health = case x[:response] do
             nil -> "unhealthy"
             _ -> "healthy"
           end
           %{state: state, health: health}
         end
       )
    |> Enum.reduce(
         %{
           "active" => %{
             "healthy" => 0,
             "unhealthy" => 0
           },
           "paused" => %{
             "healthy" => 0,
             "unhealthy" => 0
           }
         },
         fn (x, acc) ->
           put_in(acc[x[:state]][x[:health]], get_in(acc, [x[:state], x[:health]]) + 1)
         end
       )

(Ignore the inline functions, gonna clean it up after the implementation is set)

Which gives me the following output:

%{
  "active" => %{"healthy" => 1, "unhealthy" => 0},
  "paused" => %{"healthy" => 0, "unhealthy" => 1}
}

Which I then use the following way:

series = for {state, health} <- extract_labeled_data(trackers), # extract_labeled_data is the cove above
    {health, value} <- health do
  {[state: state, health: health], value}
end

Prometheus.Model.gauge_metrics(series)

To generate Prometheus metrics like:

app_gateway_tracking_sessions_count{state="active",health="healthy"} 1
app_gateway_tracking_sessions_count{state="active",health="unhealthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="healthy"} 0
app_gateway_tracking_sessions_count{state="paused",health="unhealthy"} 1

I am sure there are some things that can be improved to get from the input data to the final metrics.

Marked As Solved

mudasobwa

mudasobwa

Creator of Cure

Actually, Enum.into/3 would suffice:

|> Enum.into(%{
    [state: "active", health: "healthy"] => 0,
    [state: "active", health: "unhealthy"] => 0,
    [state: "paused", health: "healthy"] => 0,
    [state: "paused", health: "unhealthy"] => 0
  }, fn {{state, health}, v} -> {[state: state, health: health], Enum.count(v)} end)

Also Liked

mudasobwa

mudasobwa

Creator of Cure

The first step could probably be simplified to:

input
|> Enum.group_by(& {
  (if &1.pause, do: "paused", else: "active"), 
  (if is_nil(&1.response), do: "unhealthy", else: "healthy")
})

Now one might Enum.reduce/3 if zeroes are indeed required, otherwise Enum.map/2 would work:

... |> Enum.map(fn {{state, health}, v} ->
  {[state: state, health: health], Enum.count(v)}
end)
#⇒ [
#     {[state: :active, health: :healthy], 1},
#     {[state: :paused, health: :unhealthy], 1}
#  ]
mudasobwa

mudasobwa

Creator of Cure

At least you might be slightly more DRY:

Where Next?

Popular in Questions Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
wernerlaude
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
New

Other popular topics Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement