This is what http_cache_store_disk and http_cache_store_memory do when cluster_enabled is set to true: they exchange cached responses by using distributed erlang and store them either in ETS table or on disk (and in this latter case, metadata is still stored in memory).

They take into account that nodes can have different requirements such as available disk space or memory and therefore they handle cached response autonomously. A cached response can be discarded from one node and still be available on another.

If you want to cache HTTP responses from Phoenix / Plug, you can take a look at plug_http_cache.

What about running separate caches on both instances? You will waste a little bit more ram, however you will get rid of a lot of complexity, which IMO is perfect, as ram is dirt cheap these days.

It’s possible. You could always build a http endpoint around ets and server http requests that way. But none of that comes “built in” and I doubt it’s worth it in the general case.

On the beam you also have mnesia which allows you to share kind of an ets table across the cluster, but it’s not quite straight forward to use. That’s local state synchronized across the cluster and not accessing one shared resource.

Without much detail on why you need a global cache I’d probably suggest staying with redis, lacking any reasonable means of doing a tradeoff with other options.

Is fetching a cached value from another node really worth it? Is it crushingly faster?

I have my doubts. Would love to see actual metrics.

In the first 20 seconds after an http_cache_store_{disk,memory} starts, whenever another node shows up, a warm_me_up message is sent to the node that shows up, and the latter sends the n hottest stored HTTP responses.

Result will depend on your deployment type. If you’re doing rolling deployment then nodes still up will feed new nodes with HTTP responses, and the former nodes will benefit from the same mechanism once they are eventually restarted, receiving back the HTTP responses they sent to the first restarted nodes.