Questions & Help>Questions

Slow recurring insert Oban query caused by too many cancelled scheduled jobs

#oban-pro#possible-bug
DaemonSnake

DaemonSnake

Slow recurring insert Oban query caused by too many cancelled scheduled jobs

Hello,

We were having some issue with Oban Pro (1.4.2) in our AWS production environment (Aurora PG 16.1).

Pretty often during the day the following oban-internal query would takes the first place
in the performance analysis view in AWS with around:

  • 3+ AAS,
  • ~3s+ average latency
  • +1 call/s.

The query goes as follow:

SELECT o0."state", o0."scheduled_at" FROM "public"."oban_jobs" AS o0 WHERE (o0."meta" @> $1) AND (o0."state" != 'completed') AND (o0."id" < $2) ORDER BY o0."id" DESC LIMIT 1

We realized that we had many scheduled jobs that had been cancelled (around 200k) and were not getting pruned.
We decided to delete them and the above now well-behaves.

We believe it was caused by state != 'completed'.

What is the purpose of that query and why is ‘completed’ the only state that is excluded ?

2 comments
#oban-pro#possible-bug
0 179 2

First Post!

sorenone

sorenone

Oban Core Team

Welcome, @DaemonSnake! Thank you for bringing your question to the forum.

Definitely upgrade. There are major bug fixes and performance improvements.

How did you cancel those jobs? What is your pruning config? Cancelled jobs are normally prunable.

Are you using chains or workflows? That’s going to help us narrow it down.

Post #2

Where Next?

View thread on forum (has 2 responses!)
oban-pro
possible-bug
Home Questions & Help>Questions
#oban-pro#possible-bug
0 181 2
Last post

Popular in Questions Top

openscript
Questions & Help>Questions

How to set up the Elixir SDK in Intellij IDEA with the intellij-elixir plugin?
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
17 22671 9
New
Fl4m3Ph03n1x
Questions & Help>Questions

How to test a GenServer?
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
#testing#actor-model
77 13824 19
New
vonH
Questions & Help>Questions

How do I kill a process ` #PID<0.186.0` in iex?
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
#iex#processes#how-to-question
36 21149 8
New
johnnyicon
Questions & Help>Questions

How do I use the Postgres JSONB / Postgrex JSON extension?
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
#ecto
22 17588 9
New
kostonstyle
Questions & Help>Questions

Create time with one hour plus
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
#time
33 13478 5
New
chrisalley
Questions & Help>Questions

How to describe many contexts in ExUnit without a hierarchy
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
#testing#exunit
66 16773 4
New
hpopp
Questions & Help>Questions

Access Behaviour on Structs
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
#structs
31 13547 7
New
beno
Questions & Help>Questions

Pattern match with multiple match options
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
#pattern-matching#how-to-question
29 15490 6
New
wernerlaude
Questions & Help>Questions

How to get all entries by user_id
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
/phoenix#ecto#troubleshooting
4 14281 14
New
romenigld
Questions & Help>Questions

Docker run Error loading shared library libstdc++.so.6 and libgcc_s.so.1
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
/phoenix#deployment#docker-compose
30 19932 15
New

Other popular topics Top

sorentwo
News>Announcing

Oban - Reliable and Observable Job Processing
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
/oban#ecto#postgres#job-processing
977 41022 311
New
Tee
Questions & Help>Questions

Enumerables - how does Enum.reduce work with maps
can someone please explain to me how Enum.reduce works with maps
#erlang
18 14675 13
New
ycv005
Questions & Help>Questions

BEAM file compliation error after degrading Erlang version from 22 to 21
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
/phoenix#erlang
33 15699 4
New
KronicDeth
Libraries & Projects>Libraries

IntelliJ Elixir - Elixir plugin for JetBrain's IntelliJ Platform
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
#intellij
289 35421 110
New
quazar
Questions & Help>Questions

How to encode ecto schema with Jason with all fields
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
/phoenix#json#jason
17 13698 9
New
nsuchy
Questions & Help>Questions

IEX in Windows Powershell?
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
#iex#microsoft-windows#windows#powershell
15 29309 4
New
vac
Questions & Help>Questions

Formatting a string to PEM with headers and line breaks
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
2 14555 7
New
mgjohns61585
Questions & Help>Questions

Convert a string to an integer?
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
#how-to-question
3 14200 6
New
rms.mrcs
Questions & Help>Questions

Transform a list into an map with indexes using Enum module
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
35 32607 9
New
lanycrost
Questions & Help>Questions

How To Implement if...else if...else condition
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
#how-to-question
40 51494 6
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

WyewWorks

Enabling companies to succeed by building software people love.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

WyewWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

WyewWorks

Enabling companies to succeed by building software people love.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement