ycastor

ycastor

Commanded with Extreme EventStore: How to set MaxSubscribers

Hello! I’m building an application using Commanded with the Extreme EventStore Adapter and Ecto Projections and i’m having a difficulty to run it in a clustered manner.

By default, when i create a projection the Persistent Subscription on EventStore is created with the limit of 1 subscriber per subscription, in that way, after the first node connection to the eventstore, every other node will throw a subscription_already_exists error.

I took a look on the documentation for extreme and extreme adapter and i have not found a way to increase this number, i took a look at the code and found that the SubscriptionsSupervisor for the Extreme adapter has the option Keyword.get(opts, :subscriber_max_count), but i have not found any way to set this value from the Projection or through the config file.

Does anyone know how can i increase this number?

Thanks in advance!

Marked As Solved

slashdotdash

slashdotdash

I’ve added an issue to the GitHub repo for the Commanded extreme adapter to allow configuration of the subscriber max count for persistent subscriptions.

The limit is set to 1 by default to ensure that only one event handler can be subscribed at a time. This is because event handlers usually want to process events in order. As an example, Commanded Ecto projections uses the event number for idempotency checking to ensure that events received more than once, due to at-least-once-message delivery, will only be projected once. If you were to have the same projection process running on multiple nodes subscribed to the same persistent subscription then events would be processed concurrently and likely out of order. The idempotency check would mean that any out of order events would be skipped because they would be assumed to have already been processed. The current approach is to have a single connected subscriber which guarantees event ordering.

In a multi-node deployment with distributed Erlang the :global module is used to guarantee that only one event handler process will be running within the cluster. This ensures only a single persistent subscription is connected and guarantees event ordering. If you don’t use distributed Erlang then you will have an event handler process started on each node. Unfortunately the extreme library doesn’t handle the case where connecting to a persistent subscription fails due to too many subscribers. This is a limitation that needs to be addressed. It could implement some retry mechanism when connecting to the subscription fails which would allow multiple handlers to be started but only one would connect and actually process the events. This is how the Elixir EventStore using Postgres is implemented.

Where Next?

Popular in Questions Top

tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Kagamiiiii
Student & New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New

We're in Beta

About us Mission Statement