mihailacusteanu
Clustering mnesia with libcluster Gossip strategy
I am new to mnesia and I am trying to create a cluster with libcluster Gossip strategy. It works fine, the nodes auto-connect.
Now I need to also connect the mnesia databases, but I encountered this problem.
- If a new node creates it’s own schema(database) with it’s own data, I cannot add it to the cluster because I cannot change the
:extra_db_nodesbecause the schema cookie doesn’t match. Event it’s the same table structure. - Also do yo have any suggestion about how should I deal with a network split? The network split happens, the offline node is changing the same row as the cluster and then I get an inconsistent database event. I theory I can use :rpc calls to backup the data on the splitted node (which is now online) and then restore it. but I couldn’t find any any to solve the conflict with custom code, deciding per row and not just chose between the cluster and the splitted node(the one that was offline for a while)
Regarding 1. I saw Mnesiac but It won’t work with Gossip strategy.
Can those 2 points be even solved in mnesia?
Most Liked
mihailacusteanu
hi @jchrist and thanks for your reply!
Yes, unsplit it ALMOST what I need for my case, but with the addition that I need 2 runs through the conflict list:
- first I need to collect all the conflict list and that list I want to display to the user(while having the conflict not resolved)
- user choses the “right” versions and those are propagated to the whole cluster
Unsplit will chose the latest write the second time it runs
So basically what I need is a way to prevent auto sync after a netsplit.
Because now when the erlang nodes reconnect mensia will automatically sync using the latest write.
Do you have any idea if there is something similar to what I want?
I looked over the CRDT work you suggested, it’s interesting, but it’s not related to my problem.
Also regarding the first thing I asked I understood the solution and solve it by creating the schema of a new node by copying from the existing cluster.
thanks!
jchrist
You could probably do this with a custom unsplit_method similar to the built-in ones, but I’m not sure about keeping the conflict not resolved in this case: how can you later figure out which nodes need to be merged together, especially if another partition happens?
I think one way to do it would be: copy one of the existing unsplit methods and expand it to store both sides of the conflict somewhere. The user can then later choose which one to keep and that write is propagated around the cluster.
This should be preferable to waiting for the user suggestion directly in the unsplit method: what happens if the node goes down while it’s waiting for the user to log on and change it?
Do you mean via Erlang or via the files on disk?
mihailacusteanu
how can you later figure out which nodes need to be merged together, especially if another partition happens?
A node will be on a server and the other will be on a user’s laptop. So on this node(laptop) starts it will detect the conflict and create and store the conflicts and based on that let the user choose the versions. After user chooses all the version his node(laptop) will trigger the real conflict solving.
The partitions will happen between the server and the laptop so if another partition happens the other user will solve it.
what happens if the node goes down while it’s waiting for the user to log on and change it?\
I was thinking about skip all the records the first time and then try to re trigger unsplit when I have the final versions.
But I will look over the custom unsplit_method since at the moment I not sure how this is working and this may be the actual solution
Do you mean via Erlang or via the files on disk?
I meant that initially I thought I can create schemas separately and connect them afterwards without having a bad cookie error
mihailacusteanu
I also tried to “decluster” a node, but it seems once you add a node to the cluster using :mnesia.change_config(:extra_db_nodes, [new_node]) you cannot remote it from running/stopped db nodes and it will sync with the rest of the cluster regardless of making extra_db_nodes = []
@jchrist can I do that in some way?
mihailacusteanu
The method you proposed assumes that both the cluster and node to remove are up and :mnesia.del_table_copy(:schema, node_to_remove) will fail when node_to_remove is down.
I know this is a stretch, but couldn’t the cluster node reject the communication with the node_to_remove after is down and only let it sync again after manual conflict resolution ?







