toodle
Suggestions for logging RAM high-water mark while benchmarking
I’m trying to performance-tune some big data processing code and am looking for a good way to monitor the high-water mark of RAM usage while running a function. I’ve been using the ExProf library to understand running times in different parts of my code, but this doesn’t tell me how much RAM is getting used.
If I’m not careful, running this code at scale tends to overwhelm the available RAM of my machine, at which point the BEAM seems to unceremoniously reboot. I’m trying to understand the maximum amount of RAM my code consumes while running, so that I can choose an appropriate VM size.
Any suggestions on how to do this monitoring?
Marked As Solved
D4no0
The easiest way would be to use memsup from os_mon application, it does seem to cover your specific use-case:
Periodically performs a memory check:
If more than a certain amount of available system memory is allocated, as reported by the underlying operating system, the alarm {system_memory_high_watermark,
[]} is set.
If any Erlang process Pid in the system has allocated more than a certain amount of total system memory, the alarm {process_memory_high_watermark, Pid} is set.
Popular in Questions
Other popular topics
Latest on Elixir Forum
Sponsor Spotlight
Catch errors, track performance, monitor hosts and more.
Categories:
Sub Categories:
Forums
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Develop your skills with books, videos, and courses.
Producing high quality Elixir screencasts since 2017.
Enabling companies to succeed by building software people love.
Courses that'll move you from confusion to "Aha, now I get it!"