dsnipe

dsnipe

Memory leaking with long running processes

Hello.
I have an application in production under some load and it has a memory leak.
I’m preaty new in Erlang world, anyway I tried to debug it using recon library. And it looks like the problem is with binaries.
recon_alloc:memory(allocated_types) shows me that my binary_alloc and eheap_alloc increase all the time.
Tha application is quite simple.
At boot time my application loads some data from DB and run for each record a process (GenServer) with some state (not big). At one moment there might be from 1 to 10 such long running processes. I use Phoenix endpoints to listen HTTP requests and each go through all these long lived processes with request payload in format: %{"segments": [%{"some_var" => "some bn data"}]}. I suspect, that these binaries are not garbage collected, bevcause they still may have references from those long lived processes.
Migh be that a reason? And how to solve it?

Elixir 1.4, Erlang 19.3

Marked As Solved

sasajuric

sasajuric

Author of Elixir In Action

My advice is to simply give it a try and see how it fares. Adding hibernation is fairly trivial (just append :hibernate to each return tuple). And with hibernation in place you can quickly check whether the problem is fixed (if not, then the leak likely happens someplace else). If the hibernation is fast enough for you, than cool. If not, you’ll need to think of another way.

Also Liked

sasajuric

sasajuric

Author of Elixir In Action

That might indeed be a reason. A simple fix could be to hibernate each process after they do the work on the segment. So if it’s a GenServer, you can add :hibernate at the end of each tuple you return from handle_* callbacks (e.g. {:noreply, new_state, :hibernate}). This will lead to the GC of the GenServer and thus the references to binaries will be released.

Hibernation will cause some performance penalty, but whether that’s significant or not depends on the load on the process being hibernated.

Note that you can also use :recon.bin_leak/1 to try to find out the offending processes.

rvirding

rvirding

Creator of Erlang

It might be faster to just do a gc, :erlang.garbage_collect(), as doing a hibernate does more work because it first packs the process together then unpacks it on the next message. I don’t know if there is an Elixir call to this function.

NobbZ

NobbZ

Are you sure? AFAIK elixir can’t support OTP 16, since maps have only been introduced with OTP 17.

The README of the 1.4.0 tag of the github repository clearly states:

(Elixir requires Erlang 18.0 or later)

Also I think I’ve seen some kind of matrix once which desc ribed which version of erlang was supported by which version of elixir, but I can’t find it right now.


Aside of that, you can create space leaks (not memory leaks) easily when creating substrings from strings that are longer 64 byte. Therefore it is considered good practice to :binary.copy/1 strings you have extracted and want to use for longer than a few moments.

As a rule of thumb I do copy every string I pass into another function.

dsnipe

dsnipe

My bad. I forgot that I include erlang VM into build, it is 19.3.
So, do you suggest to copy all strings in my map and then pass it futher?

NobbZ

NobbZ

Since I do not know whats the source of the strings in the map actually is, I can’t tell for sure. But if they are smaller parts of some huge string, copying might help.

But to be really sure, we had to see much more code.

Similar problems can occur when you steadily eat from an input binary and then hold a reference to the very last sub-binary somewhere.

And since this is a really complex matter, I pulled in 3 links, which I found randomly:

I’ve read the GC 19.0 article by myself about a year ago and it helped me a lot to understand how garbage collection and the BEAM are working. But I’m not sure anymore if the binheap was explained well there. Therefore I put the other 2 links as well.

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
Kagamiiiii
Student & New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

Other popular topics Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
axelson
This post is a wiki (feel free to hit the edit button near the bottom right of this post to add your own changes!) This post collects co...
239 45766 226
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement