I think one thing to realise wrt gc is that if you share data between processes then when you do a gc you have to gc all the processes and the whole heap. This means that you need a real-time collector which complicates things. Running in multiple threads which the BEAM does complicates matters even more as you need to make it thread safe and/or pay a large cost in synchronisation. This is now done for the large binaries which results that sometimes it can take long time to reclaim their memory. Basically every process which has referenced the binary has to do a full gc before the binary can be reclaimed. This can take a comparatively long time and overflowing memory with unreclaimed large binaries is actually a problem.

So while not sharing data and copying data when sending messages is actually quite a good way of doing it even if it sounds wrong. Way back when I did some implementations of erlang with shared memory and real-time collectors and it is not trivial to get it right. Fun though, but not trivial.

In 22 there are some new special memory areas which are shared so some things are good and really fast but you also pay a heavy price. Check out atomics, counters and persistent_terms and you will see what I mean.

I think you will end up copying most of the data anyway but it a much less controlled fashion. As someone has pointed out you are viewing the data sent in a message as one big coherent chunk created in the sender and sent to the receiver, but this is generally not the case. Usually the data in a message is put together from smaller bits which come from many different processes. This will mean that either you have a right mess of keeping track of what and from where, or you will be doing a lot of copying between processes which is what you are trying to avoid.

A simple example of what I mean. Assume A sends a chunk of data to B which sends it on to C, so A -> B -> C. From what I understand of your method is that the data is not copied and A knows it has sent to B and B knows it has sent it to C. What happens if/when B dies? A does not know about C so if A now dies it can’t copy the data to C. Or should B tell A about C? But that means a process has to keep track of where all its data has come from, which we don’t need to do now.

This is exactly the problem you get with atomics, counters and persistent_terms. Atomics and counters solve the problem by only allowing integers while persistent_terms allow complex data structures but require a global gc when you update the data.

yes, precisely that. However, what @marcello says is true mostly for the small messages.

If you have a message consisting of binaries larger that 64 bytes (which is, relatively small), these will not be copied and instead will be allocated somewhere in designated space on the BEAM. So, if you are sending large blobs between processes these are not copied over.

You’re missing that processes die and that makes their heap be cleaned up. Or in other words: When state is shared you need to make sure to only clean it up when nobody needs it anymore. By copying data there’s no need to track processes, which need access to a piece of data.

This is what we mean by garbage collection. In order to do this globally you have to stop EVERY process when doing a garbage collection, like Java does. This is a problem.

Notably, the actual runtime will mutate underlying data in certain cases if it can be sure that the old data is no longer accessible in order to improve performance. These optimizations would become hard or impossible if data liveliness has to be tracked across all possible processes that can touch the data.

Related to what peerreynders said, having each process with it’s own heap ensures that when it’s time for a particular process to do work, all of it’s data is physically co-located in memory, which makes it easier for the CPU caches to work as designed. In fact many processes are small enough to fit inside the CPU caches entirely, which dramatically speeds up performance.

Really though, it’s all about garbage collection, either in the normal case or in the process death case. If processes can share data you either need to impose a ref counting penalty on ALL data or you need to impose “stop the world” garbage collection. Refcounting large binaries is an acceptable trade off for that data type, but not an acceptable trade off for others. Stop the world garbage collection isn’t acceptable for a soft real time system. Ergo, isolated process heaps.