markmark206

Doing something every few hours, the simplest approach?

I would like to perform an action periodically, and I am looking for the simplest reasonable way to do this.

As an example, let’s say I want to delete old entries from a db table, every few hours. The action is idempotent, relatively inexpensive, precision “doesn’t matter,” and concurrent executions of multiple runs (e.g. from multiple replicas of the service) is not a problem (db transactions will handle them safely).

A commonly recommended approach for doing this seems to be a variation of using a GenServer with send_after (or, I suppose, spinning up an oban job;).

This makes a lot of sense, but I coded up just running an infinite supervised “do”+“sleep” recursion, and it seems to work, and it seems ridiculously concise and simple, and I can’t explain to myself why that wouldn’t be enough.

Is there any reason why I shouldn’t do this? ; )

If you have any thoughts / guidance on this, I would much appreciate them!

Thank you!

PS An example of what this might look like in code:

application.ex (starts the task, restart: :permanent):

defmodule MyApp.Application do
    def start(_type, _args) do
        children = [
           ...,
           Supervisor.child_spec(
               {Task, fn -> MyApp.Sweeper.delete_old_data(sleep_ms) end},
               restart: :permanent
           ), 
           ...
        ]
        Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor)
    end
end

where delete_old_data() just keeps doing the thing and sleeping, forever:

defmodule MyApp.Sweeper do
    def delete_old_data(wait_ms) do
        ... delete old things ...
        Process.sleep(wait_ms)
        delete_old_data(wait_ms)
    end
end

16 comments

#scheduled-tasks

30 1503 16

2023-03-05 09:33:36 UTC

Most Liked

sasajuric

Author of Elixir In Action

If you want the shortest possible out-of-the-box solution with no external dependency, there’s :timer.apply_interval.

Such solution should be no worse than a custom GenServer +send_after.

My preferred approach though is a GenServer which starts the task as a child process. It also has to be one GenServer per each periodic job, so I can inject each job in the proper place in a supervision tree. This is a big reason why I don’t like and avoid quantum & similar libs.

Instead I wrote my own periodic abstraction which follows the principles outlined above. I blogged a bit about it here.

Post #12

kwando

It certainly works like you did but I think it is but I is more idiomatic to keep the details out of the application file. The very least you could do is to put the child_spec/1 inside the MyApp.Sweeper module…

defmodule MyApp.Application do
    def start(_type, _args) do
        children = [
           ...,
            MyApp.Sweeper, 
           ...
        ]
        Supervisor.start_link(children, strategy: :one_for_one, name: MyApp.Supervisor)
    end
end

defmodule MyApp.Sweeper do
  def child_spec(_) do
     Supervisor.child_spec(
       {Task, fn -> MyApp.Sweeper.delete_old_data(sleep_ms) end},
         restart: :permanent
       )
   end
end

but really I think a standard GenServer + send_after / :timer.send_interval is the way to go for this… easier to read and understand

defmodule MyApp.Sweeper do
   use GenServer
   def init([]) do
     :timer.send_interval(self(), :timer.hours(5), :delete_old_data)
     {:ok, []}
   end

  def handle_info(:delete_old_data, state) do
     # delete old data here, or spawn a Task doing it to keep the Sweeper responsive
   {:noreply, state}
  end

  def start_link([]) do
    GenServer.start_link(__MODULE__, [])
  end
end

Post #9

dimitarvp

Seconding @sasajuric and @BartOtten here, just roll your own GenServer per task – it’s just 10-20 coding lines of boilerplate maximum.

Or use Periodic. It’s a very small and functional thing (last I used it at least, which was like 3 years ago). Or you can rip out the code you need from it because again, it’s very small and works fine.

I would make a few GenServers though. It’s a one-time investment that can pay huge dividends. If you want to get slightly fancy maybe you can store “when was the last time task X ran” in a small file; though you mentioned that it’s not critical if a task gets executed a bit more rarely every now and then (when the app is rebooted) so if that’s indeed the case then it’s probably best to not bother with keeping state outside of memory.

Post #14

cmo

I would use a GenServer at least so you have it in a file somewhere and not in application.ex. Not sure what you’d gain from going the task or process method? Are you trying to reinvent a wheel or save some LOC? It certainly makes it more work to extend and hides the code away.

If you had oban as a dep already that would probably be the right choice.

Post #7

tfwright

Not sure what you’d gain

Well, he did say he was aiming at simplest, and I’d actually tend to agree that a module not importing a separately defined behavior is a bit simpler…at least in principle? I’m also interested in anything more substantial he might lose though by starting here.

For myself I usually start with Quantum unless I know I’m going to need persistence. I do always know I am going to need scheduling.

Post #8