samkit
Acquire locks in ets table row in order they were requested
I have n number of identical rabbitmq stream consumers with prefetch count set to 1. All the messages have a tag, lets suppose user_id in this case. The messages with the same user_id tag must be processed in order they arrive in the stream.
The consumer fetches the state corresponding to the tag from ets, acquires a lock ,does processing, update the row and release the lock. Now, lets suppose consumer 1,2,3 all have messages with the same tag while consumer 2 and 3 are trying to acquire the lock in a loop and consumer 1 having the lock. How do I ensure that consumer 2 gets the lock first/message 2 is processed first after consumer 1 has released the lock.
Thanks for reading:)
My current implemetation has a single consumer consuming all the messages with the same tag, therefore order is maintained automatically, but in this case I need to have as many consumers as the number of different tags/users on the platform which doesnt scale out very well.
Most Liked
al2o3cr
This talk walks through a very similar problem statement:
Short short version: it presents an algorithm for routing RabbitMQ messages so that ones that need consistent ordering are always processed by the same consumer so (for instance) the ones with user_id = 1 are seen in submission order.







