artem

artem

Commanded/CQRS: how to project "gluing/merging" into the existing projection

I am learning Commanded and CQRS in general with a small pet project that is essentially yet another wrapper over a ChatGPT conversation with the prompts/buttons specific to the use case.

I have created a Conversation aggregate and pretty significant part of events is going to be something like ConversationStepAdded.

**Read model**
On the read side in one of the read models I’d want to get a nicely denormalized conversation structure (a nice feature of CQRS!), something like:

- conversation_id
- conversation_title
- some_other_metadata
- a_list_of_conversation_steps
  - conv_step_type // e.g. user or assistant related
  - conv_step_text
  - some_more_step_related_data

Ideally this whole structure will be an Ecto schema with a_list_of_conversation_steps being an embedded_schema (via embeds_many) that is probably going to be mapped to Postgres JSONB field.

**Problem: how to update a_list_of_conversation_steps from a single step event?**
Looking at how commanded projectors work (and CQRS in general?) projection is to be build one event at a time without seeing an aggregate.

Then how would I update a_list_of_conversation_steps when ConversationStepAdded arrives? Since I am learning, I will highly appreciate not only answer to this particular question, but general guidance as well.

  • Should I make ConversationStepAdded event carry not just the step being added, but whole list of steps accumulated by now to rewrite whole a_list_of_conversation_steps field via SQL UPDATE?
  • Should I go deeper SQL/Ecto and create a complex Ecto.Multi changeset to append a to a_list_of_conversation_steps’es embedded schema and then Ecto will somehow map it to proper SQL (I guess somewhere on the way I will still need to learn which exactly SQL can append to JSONB field)?
    • I guess it could work in this particular case, but not so nicely e.g. if a_list_of_conversation_steps would be just a long string without explicit embedded_schema
  • Or is it possible (and recommended?) to have some elixir code there to read the current projection, operate on it in Elixir level (well, still via changeset I suppose) and UPDATE it back to database?
  • Or something very different? Maybe the whole approach is wrong and there is a way for this problem not to exist in the first place?

What would you do in such a situation?

Marked As Solved

slashdotdash

slashdotdash

In a read model projector you can fetch existing data that has already been projected and update it. So in this example when projecting the ConversationStepAdded event you could fetch the existing Conversation from the database and then add the new conversation step to the list of conversation steps before saving the updated model. You can use the Ecto.Multi.run/3 function to fetch and update an existing model in a Commanded Ecto projector.

It’s not really necessary to use Ecto changesets in a read model projector for validation purposes since the domain events are the source of truth in an event sourced application. The projector cannot reject an event that has already happened. Its job is to represent the application state in a format optimised for querying needs, as described by the events that it receives.

Also Liked

slashdotdash

slashdotdash

A projector reading from the table(s) that it “owns” and is projecting into is absolutely fine and recommended.

A projector is a specialised event handler which means by default it receives events from the globally ordered $all event stream. Which guarantees that events are always received in the order they were persisted to the eventstore. The same order for all event handlers that are configured to receive all events.

Since event handlers (projectors) run independently they are not guaranteed to be runing in sync, one slow handler does not hold up any others. Therefore it is not safe to query tables that are populated by another projector. The other projector could be lagging behind and have not seen the event(s) that the second projector expects to have been projected. To resolve this you could have a single projector to project into all the affected tables, or you project the subset of person info needed by the projector that needs to query it, or you accept that there might sometimes be stale data for querying that has be resolved somehow.

Where Next?

Popular in Questions Top

openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
mathew4509
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New

We're in Beta

About us Mission Statement