nezzart

nezzart

Periodic tasks in a single GenServer -- not everything yet makes sense

I’ve read on GenServer, but am not able yet to use it because not everything yet makes sense to me.

I want this:

  • when a user clicks on a button, a new task begins
  • a task terminates depending on some external condition, say, an external rest-api webservice has returned “terminate = true”
  • if a task is already running, clicking on a button shouldn’t begin a new
  • a task poll an external rest-api webservice every 30 seconds

I was adviced to use a single GenServer. My questions:

  1. how can I associate a new task or its name with a user’s id? so that when he clicks on a button I can check if I need to create a new task or do nothing if there’s already one running?

  2. how can I add/remove an item to MyWorker? Meaning, should I keep a list of users id?

  3. if there’re no tasks, GenServer should stop. I think. Is it a wise thing to do? If so, how can I stop/run it?

  4. to re-run a task every 30 seconds I need to call schedule_work() in “poll_external_service”, correct? Thus, to terminate it, depending on a response from an external web rest service, I merely don’t run schedule_work(), correct?

    defmodule MyWorker do
    use GenServer

     def start_link do
       GenServer.start_link(__MODULE__, [], name: :my_app_worker)
     end
    
     def init(state) do
       schedule_work() 
       {:ok, state}
     end
    
     def add_item(x) do
    
     end
    
     def remove_item(x) do
    
     end
    
    
     defp poll_external_service do
       # a request to an external web server
    
       schedule_work()
     end
    
     defp schedule_work() do
       Process.send_after(self(), :work, 30 * 1000) 
     end
    

    end

Marked As Solved

net

net

I may have misunderstood what you’re trying to do, but this might help.

Assuming that you only wanted to store the user ID for each ask, I used a MapSet. If you want to store additional data with the task (such as a specific URL to poll), you should use a Map (with the key being the user ID, as map keys are unique).

In process_tasks/1 we use Enum.filter/2 to both call a function for each task, and to remove tasks that the external API has indicated should be terminated. The case ExternalAPI.poll(id) do is of course, a contrived example. What’s important is that the anonymous function given to Enum.filter/2 returns false if the task should be removed. (You can give Enum.filter/2 a map too, if you go that route. In that event the anonymous function should take a tuple {user_id, data} representing each key-value pair in the map.)

There’s no need to stop the GenServer when there are no tasks. Erlang processes are very lightweight.

defmodule MyWorker do
  use GenServer

  @tick_interval 30_000

  def start_link, do: GenServer.start_link(__MODULE__, nil, name: __MODULE__)

  def add(id), do: GenServer.cast(__MODULE__, {:add, id})

  def remove(id), do: GenServer.cast(__MODULE__, {:remove, id})

  def init(_) do
    tick()

    # Here we use a MapSet as it will reject duplicate items.
    # If you wish to store more data with a task than just a user ID,
    # use a Map instead.
    {:ok, MapSet.new}
  end

  defp tick, do: Process.send_after(self(), :tick, @tick_interval)

  def handle_cast({:add, id}, tasks), do: {:noreply, MapSet.put(tasks, id)}

  def handle_cast({:remove, id}, tasks), do: {:noreply, MapSet.delete(tasks, id)}

  def handle_info(:tick, tasks) do
    tasks = process_tasks(tasks)

    tick()

    {:noreply, tasks}
  end

  defp process_tasks(tasks) do
    tasks_list =
      Enum.filter tasks, fn(id) ->
        # Here we process the individual task, such as polling
        # the external API.
        case ExternalAPI.poll(id) do
          # We return true from this anonymous function if the task should
          # stay, and false if the task should be removed.
          "terminate = true" -> true
          "terminate = false" -> false
        end
      end

    # `Enum.filter/2` returns a list, so we must convert it back into a MapSet.
    MapSet.new(tasks_list)
  end

  def handle_info(_, tasks), do: {:noreply, tasks}
end

Also Liked

OvermindDL1

OvermindDL1

Yes, and it is described in the GenServer documentation. :slight_smile:

Imagine genserver implemented this way in pseudo-code:

def start_link(user_module, args) do
  {:ok, spawn_link(fn -> gen_server_init(user_module, args) end)}
end

def gen_server_init(user_module, args) do
  {:ok, state} = user_module.init(args)
  gen_server_loop(user_module, state)
end

def gen_server_loop(user_module, state) do
  receive do
    {:__special_genserver_cast__, msg} ->
      {:noreply, newstate} = user_module.handle_cast(msg, state)
      gen_server_loop(user_module, newstate)
    {:__special_genserver_call__, msg, from} ->
      {:reply, outmsg, newstate} = user_module.handle_call(msg, from, state)
      send(from, outmsg)
      gen_server_loop(user_module, newstate)
    unknown_msg ->
      {:noreply, newstate} = user_module.handle_info(unknown_msg, state)
      gen_server_loop(user_module, newstate)
  end
end

And so forth, but with significantly more error checking and cases and such. GenServer is just an infinite receiving loop passing the state from iteration to iteration. :slight_smile:

net

net

You’re right, that is better. Instead of Enum.filter/2 you might consider just using Enum.each/2 and calling MyWorker.remove/1 from the process handling the individual task if the task should be removed.

Qqwy

Qqwy

TypeCheck Core Team

The MyWorker uses the :name option as part of the start_link call, which means that the process is registered under a global (node-wide) name. The other calls that use GenServer.call and GenServer.cast use the same name to access it later.

So this means that the process will remain available after starting until your application closes, the process crashes, or you call GenServer.stop/2 from somewhere. In any case, the process will remain regardless of the connecting/disconnecting of users to your web-facing interface.

balena

balena

I would recommend you to have a look at the Registry module.

First, some considerations:

  1. Say you run one GenServer per user;
  2. And you have a rescheduler as a separate GenServer process.

Tackling the first part, which involves associating a user’s id with a process, start one if one is not running and keeping a list of executing processes based on the user’s name, can be done using the following construct:

defmodule MyWorker do
  use GenServer

  @doc """
  Should be called before `start_worker`.
  """
  def start_link() do
    Registry.start_link(:unique, __MODULE__)
  end

  @doc """
  Starts a worker process for a given `user` to handle the given `task`.
  """
  def maybe_start_worker(user, task) do
    case Registry.lookup(__MODULE__, user) do
      [{pid, _value}] ->
        {:ok, pid}  # do nothing, already running
      [] ->
        name = {:via, Registry, {__MODULE__, user}}
        GenServer.start_link(__MODULE__, task, name: name)
    end
  end

  def init(task) do
    self() |> Process.send_after(:process_task, 0) 
    {:ok, task}
  end

  def handle_info(:process_task, task) do
    # do something with the task
    access_external_service(task)

    # finishes the process, nothing else to do for the moment
    {:stop, :normal, task}
  end

  defp access_external_service(task) do
    # a request to an external web server
  end
end

Assuming the above, then now you could create another process to re-run the worker process every 30 seconds. This another process can hold a simple Map where the key is the user’s id and the value is the task. Just call MyScheduler.add_item/2 when the user clicks on the button, followed by MyWorker.start_worker/2 if you want to process the task immediately, otherwise, the worker will be executed 30 seconds later by the scheduler.

defmodule MyScheduler do
  use GenServer

  def start_link() do
    GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
  end

  def add_item(user, task) do
    GenServer.call(__MODULE__, {:add_item, user, task})
  end

  def remove_item(user) do
    GenServer.call(__MODULE__, {:remove_item, user})
  end

  def init(map), do: {:ok, map}

  def handle_info({:reschedule, user, task}, map) do
    MyWorker.maybe_start_worker(user, task)
    timer = self() |> Process.send_after({:reschedule, user, task}, 30_000)  # reschedule
    {:noreply, %{map | user => {task, timer}}}
  end

  def handle_call({:add_item, user, task}, map) do
    timer = self() |> Process.send_after({:reschedule, user, task}, 30_000)
    {:reply, :ok, map |> Map.put(user, {task, timer})
  end

  def handle_call({:remove_item, user}, map) do
    map =
      map |> Map.get_and_update(user, fn {_task, timer} ->
        timer |> Process.cancel_timer()
        :pop
      end)
    {:reply, :ok, map}
  end
end

With the above:

  1. Say your process takes more than 30 seconds to execute; suppose 45 seconds. Then, when the scheduler gets the first timer event, it won’t start a new process at that moment; as it keeps retrying every 30 seconds, the task will be executed in the next try, after 60 seconds.
  2. In order to stop rescheduling, execute MyScheduler.remove_item/1 for the same user ID you used in MyScheduler.add_item/2.

Where Next?

Popular in Questions Top

albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
wernerlaude
In AR this is so simple @articles = current_user.articles How to do in Ecto? def index(conn, _params) do current_user = conn.assig...
New

Other popular topics Top

sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New

We're in Beta

About us Mission Statement