Yes, something like this should work:

config :kernel,
  net_ticktime: 120

If you’re talking about pubsub then forking is not needed. Pubsub already supports adapters to use different backends. E.g. there’s a redis based backend for heroku users.

It appears that nodes just lose connection from time to time

This is what global source code says about it:

%% ----------------------------------------------------------------
%% Prevent Overlapping Partitions Algorithm
%% ========================================
%%
%% 1. When a node lose connection to another node it sends a
%%    {lost_connection, LostConnNode, OtherNode} message to all
%%    other nodes that it knows of.
%% 2. When a lost_connection message is received the receiver
%%    first checks if it has seen this message before. If so, it
%%    just ignores it. If it has not seen it before, it sends the
%%    message to all nodes it knows of. This in order to ensure
%%    that all connected nodes will receive this message. It then
%%    sends a {remove_connection, LostConnRecvNode} message (where
%%    LostConnRecvNode is its own node name) to OtherNode and
%%    clear all information about OtherNode so OtherNode wont be
%%    part of ReceiverNode's cluster anymore. When this information
%%    has been cleared, no lost_connection will be triggered when
%%    a nodedown message for the connection to OtherNode is
%%    received.
%% 3. When a {remove_connection, LostConnRecvNode} message is
%%    received, the receiver node takes down the connection to
%%    LostConnRecvNode and clears its information about
%%    LostConnRecvNode so it is not part of its cluster anymore.
%%    Both nodes will receive a nodedown message due to the
%%    connection being closed, but none of them will send
%%    lost_connection messages since they have cleared information
%%    about the other node.
%%
%% This will take down more connections than the minimum amount
%% of connections to remove in order to form fully connected
%% partitions. For example, if the user takes down a connection
%% between two nodes, the rest of the nodes will disconnect from
%% both of these nodes instead of just one. This is due to:
%% * We do not want to partition a remaining network when a node
%%   has halted. When you receive a nodedown and/or lost_connection
%%   messages you don't know if the corresponding node has halted
%%   or if there are network issues.
%% * We need to decide which connection to take down as soon as
%%   we receive a lost_connection message in order to prevent
%%   inconsistencies entering global's state.
%% * All nodes need to make the same choices independent of
%%   each other.
%% 
%% ----------------------------------------------------------------

So, the real question is are you sure that the connection between nodes is not lost from time to time? If you’re sure, what tool do you use to check it?

Also very interested in the ideal global config.

We see a lot of packet loss between regions on Fly depending on the region and have managed to only get 4 regions (8 nodes) running somewhat reliably.

Had to build latency monitoring. See: Status - Supabase Realtime

It logs latencies to our logging infra too if they are over a threshold.

Note: the default PubSub adapter sends all messages to all nodes.

The Redis adapter would route everything through one box which would negate the whole global cluster idea.

I am @chasers !

I think the way Partisan could help in this situation is exactly what @chasers suggests. However, this can only be accomplished when using the HyParView topology which maintains a partial mesh (eventually and probabilistically) forming a fully connected graph and uses transitive message transmission.

Using HyParView rules out the possibility of using partisan’s OTP behaviours at the moment (working on trying to solve this next, the key issue is how to implement monitoring).

Also notice that using Partisan rules out using Phoenix as it relies on disterl and OTP. For Phoenix to work we would need to fork it and teach it how to use Partisan and Partisan’s OTP behaviours.

I have plans to test Partisan itself and Bondy (which uses Partisan) in Fly.io for this very same reason.