vrod
Monitoring messages in process mailboxes
I am wondering for advice on how to monitor an application with many genservers doing long-running work. Often we want to know “how many messages are in the queue?” The process has a mailbox not a queue but this is the same idea : how can we see what our app is doing? I have seen GenStage.estimate_buffered_count/2 and I am wondering if that is the best way. There is a mix of genserver and genstage, so I thought maybe Process.info/1 could help with genservers, but I noticed that this function seems to require a pid and it will not work with a process name. Also I do not see the messages in the process info? Only message_queue_len?
Another idea was because we are using pubsub, maybe we could add subscribers that would count messages but that feels maybe smelly.
Thank you for suggestions!
Marked As Solved
rvirding
A simple way of of finding the length of the message queue is Process.info(pid, :message_queue_len) which avoids copying the whole message queue. Only having :message)_queue_len in the default info is to NOT copy all the messages unless you explicitly request it.
Also Liked
nathanl
I haven’t used this yet, but as of OTP 27.0, you can subscribe to notifications about mailbox lengths.
:erlang.system_monitor(self(), [{:long_message_queue, {disable, enable}}])
disable and enable are integer thresholds for when you want receive a message. More explanation is in the docs.
rvirding
It is not really the handle_info callback which processes the actual message. The GenServer has a top-loop which sits and receives messages arriving at the process. When a message arrives it checks the message format and and calls the relevant callback to handle the message. When the callback returns it goes back into the top-loop and waits for and then processes the next message aand then waits for the next one and then … . In your case it calls the handle_info callback with the message.
This means that there is no real buffering of messages in the GenServer message queue so checking its length will generally return a small number. The only time the queue can become long is if the GenServer becomes overloaded.
RudManusachi
To find a pid by process name we could check Process.whereis(:name)
And now we could see the messages via Process.info(pid, :messages), but only be careful, as this operation copies all messages to the process that called. So if mailbox is huge - the operation might be expensive.
mpope
A way to get all ‘monitor-able’ processes is to register them in a process group under the :pg module at startup. Then you’ll have a list of pids to call Process.info/1 on.
:pg.join(:message_length_procs, self())
lud
If you need to do that kind of stuff a general patter is to do the actual work in a side process of the gen server (a Task process managed by the gen server), while the gen server just accept messages and put them in a queue (with the :queue module). So you can do whaterver you want with the queue : inspect it, filter it, or use another queue with priorization, etc.







