hubertlepicki

hubertlepicki

Profiling memory usage in given process

I’ve got some requests misbehaving, and processes handling them suddenly allocate a lot of memory, which can crash my Beam VM.

I can monitor and detect that, I can also enforce max memory usage by process, that’s not an issue.

I would like to, however, know what is eating up the memory. Ideally I’d be able to take a snapshot of memory used by current process, have it dumped to a file that contains Erlang terms, which later I could analyse and find out what’s on stack, what’s on heap etc.

Is there something out there I can use to achieve that?

Most Liked

sasajuric

sasajuric

Author of Elixir In Action

The stack trace should help you narrow down the problematic code. We actually had a similar situation recently, and we ended up periodically logging the processes with unusually high mem usage, together with their stack traces. In this particular situation that info was enough to completely understand the root cause. I agree that it’s not quite what you’re looking for, but it can help you in understanding the problem.

It’s also worth keeping in mind that in a general case excessive mem usage can be caused by other things, such as messages accumulated in the process queue, or refc binaries, or ets tables, so finding the cause is not necessarily as straightforward as logging the current heap of the process, which is why I mentioned a couple of different things in my previous answer.

garazdawi

garazdawi

Erlang Core Team

That is true. However, since you cannot loop without calling functions I’ve very rarely found that to be a limiting factor.

axelson

axelson

Scenic Core Team

You can run observer locally and connect to a remote node to view its information. Here’s a blog post about it: http://jbavari.github.io/blog/2016/03/11/using-erlang-observer-on-a-remote-elixir-server/

sasajuric

sasajuric

Author of Elixir In Action

AFAIK, it’s not possible to get what you’re looking for out of the box. To get some clues, you could use a combination of the following:

  • memory usage of the process
  • process stack trace
  • message queue length
  • total binary memory usage
  • total ETS memory usage

In case you don’t know which process is the offender, you might also need to include :initial_call and :registered_name (both available via Process.info).

Coupled with the knowledge of the code, this should help you narrow down the problem.

hubertlepicki

hubertlepicki

I already solved the issue as in why and what is happening. It was this bug in absinthe we stumbled upon: https://github.com/absinthe-graphql/absinthe/issues/569

I’d like to know if there’s something that’ll help me out next time tracking down such and similar issues :slight_smile:

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
New
gazoon
I want to know absolute current module path. In python i could do that: os.path.abspath(__file__) Does elixir have anything similar?
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
gonzofish
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement