Questions & Help>Questions

Receiving & processing events via incoming Webhooks

#webhooks#queues#events#pipelines
Essstp

Essstp

Receiving & processing events via incoming Webhooks

I’m an Elixir developer (relatively new) from Australia and I am loving it :smile:

I’m working on a SaaS startup which is an industry-specific / niche Learning Management System (LMS). Courses will be launched and played externally (by multiple 3 party solutions) before returning to my system. However, I need to retrieve and process course progress events (course started, course module completed, course completed etc.) for all courses / students on my platform, from the 3rd party services (whilst course are in progress). If the startup is successful, this will end up being a lot of events coming in via webhooks, all the time!

I realise the default answer is probably something along the lines of this: Webhook Service @ PagerDuty

But I am wondering if we should looking at utilising Amazon SQS (our app is already hosted in AWS) with the new Broadway library? I am a little confused about which of GenStage, Flow and Broadway would best fit my use case. Appreciate any thoughts/advice on this. Thanks!

4 comments
#webhooks#queues#events#pipelines
8 1798 4

Most Liked

Essstp

Essstp

Wow…I am in Coffs Harbour too!
I thought there were only two Elixir devs in town (me and another guy I know).
Now that there is three of us, we’re going to have to start up our own regional NSW meetup group. Haha.
We should catch up one day…

Thanks for the feedback on my question.

3
Post #3
sanswork

sanswork

Where in Australia are you based? I’m up in Coffs harbour area but will be at the Melbourne meetup on the 21st if you’re down that way.

The work load on each hook sounds pretty low so personally I’d just process them in a normal phoenix request flow and plan to scale horizontally at that point. That should get you really far and is easy to scale horizontally.

1
Post #2

Where Next?

View thread on forum (has 4 responses!)
webhooks
queues
events
pipelines
Home Questions & Help>Questions
#webhooks#queues#events#pipelines
8 1800 4
Last post

Popular in Questions Top

sergio
Questions & Help>Questions

Updating a field using Ecto one-liner?
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
15 14579 3
New
Tee
Questions & Help>Questions

Enumerables - how does Enum.reduce work with maps
can someone please explain to me how Enum.reduce works with maps
#erlang
18 14675 13
New
_russellb
Questions & Help>Questions

Web scraping tools
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
#web-scraping
123 19127 45
New
dokuzbir
Questions & Help>Questions

How to convert list to string without losing brackets
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
26 14426 10
New
stefanchrobot
Questions & Help>Questions

How to decode a JSON into a struct safely?
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
#structs#json
29 20780 26
New
johnnyicon
Questions & Help>Questions

How do I use the Postgres JSONB / Postgrex JSON extension?
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
#ecto
22 17588 9
New
Patoshizzle
Questions & Help>Questions

(Postgrex.Error) FATAL 28P01 (invalid_password) password authentication failed for user “postgres”
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
#ecto#postgres#troubleshooting
10 28696 20
New
Exadra37
Questions & Help>Questions

It's possible to write a custom Guard to check for a blank string?
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
#guard-clauses#strings#guard#binary
20 14093 21
New
belgoros
Questions & Help>Questions

Regex question for hyphen match
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
#learning-elixir
9 12939 10
New
Fl4m3Ph03n1x
Questions & Help>Questions

Learning Elixir, frst impressions ( plz don't kill me ! )
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
#learning-elixir
371 25801 61
New

Other popular topics Top

Qqwy
Blogs & Podcasts>Blog Posts

Elixir Blog Posts
Update: How to use the Blogs & Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
hexdocs.pm
#blog-posts#wiki#stickies#official-blog-posts-thread
3268 119930 1237
New
Brian
Questions & Help>Questions

Import a module from a file into IEX
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
#iex
35 32139 16
New
joaquinalcerro
Questions & Help>Questions

What is the best approach for fetching large amount of records from postgresql with ecto
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
#ecto
126 24587 27
New
jerry
Questions & Help>Questions

Ecto query using like/ilike in query
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
#ecto
17 16504 10
New
polypush135
Questions & Help>Questions

Proper use of Task.async?
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
#task
26 15568 8
New
script
Questions & Help>Questions

Removing spaces and characters from string
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
#regex
11 14427 2
New
stefanchrobot
Questions & Help>Questions

How to decode a JSON into a struct safely?
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
#structs#json
29 20780 26
New
qwerescape
Questions & Help>Questions

How to get the call stack/stack trace at any point in code?
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
53 15772 4
New
msaraiva
Your Libraries & Projects>Libraries

Surface - A component-based library for Phoenix LiveView
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
/phoenix#liveview#livecomponents#surface
564 42633 214
New
skosch
Questions & Help>Questions

Put/update deep inside nested maps (and auto-create intermediate keys)
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
#data-structures#maps#immutability
52 20047 11
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

Coders51

We build reliable cloud platforms for business-critical systems.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

WyewWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

WyewWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement