hubertlepicki

hubertlepicki

Telemetry - calculating custom metrics like RPM (requests per minute) on the host itself

So I have been using Telemetry for a while, but on hosts themselves been mostly doing simple metrics that are counters / last_value, and report to statsd via a simple reporter. Nothing fancy, and the actual aggregation mostly happens outside application.

I have another project, however, where we want to calculate some of these metrics on host itself. The reason being of havin millions events per minute that we do not really care about in any other way than let’s say “give me number of events per minute emitted”. So we don’t want to flood the external system with the irrelevant data and pre-calculate the values on Elixir application, and send aggregate to a reporting dashboard via API call.

I’m not sure if I am doing it right, however. Let’s simplify the use case a bit, let’s say I am interested in a metric of “throughput”, defined as “number of requests served in given minute”.

I am currently using a combination of two metrics to achieve that. First, I have “counter()” that simply increments number of requests served by the application.

Then, I have a custom poller, a GenServer, which wakes up every 60 seconds, stores the current counter value in it’s state, and subtract previous value from current value, and emits “last_value()” type of metric.

This last_value() is my “throughput” that I report to the dashboard using custom reporter.

Is this the right way to do it?

Marked As Solved

hauleth

hauleth

I would say that using just counter should be enough. Then you have methods for calculating the derivative in time in storage. For sure Prometheus, InfluxDB and DataDog have these, so I assume that other backends support something like that as well.

Also Liked

hubertlepicki

hubertlepicki

An yes, right, I can do that in fact. Damn, that’s too simple :sweat_smile:

Thanks!

Where Next?

Popular in Questions Top

itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers' Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement