hubertlepicki

hubertlepicki

Telemetry - calculating custom metrics like RPM (requests per minute) on the host itself

So I have been using Telemetry for a while, but on hosts themselves been mostly doing simple metrics that are counters / last_value, and report to statsd via a simple reporter. Nothing fancy, and the actual aggregation mostly happens outside application.

I have another project, however, where we want to calculate some of these metrics on host itself. The reason being of havin millions events per minute that we do not really care about in any other way than let’s say “give me number of events per minute emitted”. So we don’t want to flood the external system with the irrelevant data and pre-calculate the values on Elixir application, and send aggregate to a reporting dashboard via API call.

I’m not sure if I am doing it right, however. Let’s simplify the use case a bit, let’s say I am interested in a metric of “throughput”, defined as “number of requests served in given minute”.

I am currently using a combination of two metrics to achieve that. First, I have “counter()” that simply increments number of requests served by the application.

Then, I have a custom poller, a GenServer, which wakes up every 60 seconds, stores the current counter value in it’s state, and subtract previous value from current value, and emits “last_value()” type of metric.

This last_value() is my “throughput” that I report to the dashboard using custom reporter.

Is this the right way to do it?

Marked As Solved

hauleth

hauleth

I would say that using just counter should be enough. Then you have methods for calculating the derivative in time in storage. For sure Prometheus, InfluxDB and DataDog have these, so I assume that other backends support something like that as well.

Also Liked

hubertlepicki

hubertlepicki

An yes, right, I can do that in fact. Damn, that’s too simple :sweat_smile:

Thanks!

Where Next?

Popular in Questions Top

bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
AstonJ
We’ve put together this wiki for Phoenix LiveView - please feel free to add any info you feel is worth including. What is Phoenix LiveV...
New

We're in Beta

About us Mission Statement