KekKekington

KekKekington

Best practices when working with periodic tasks

I was tasked with a low-priority project at work that will be used to monitor any websites/services belonging to our clients (mantained by us) and figured that it could be a good time to show the benefits of using Elixir. My thought was that since this is a very simple task, I could focus on following the best practices and as a practical excercise for me, since I always read/listen about Elixir but don’t have much time to actually write it…

What I have planned so far is a supervised GenServer that routinely calls itself to execute the task every X seconds, where the task is performing a GET at /healthz and looking for a 200 reply or a basic ping if I get a 404 there. That task would run that function for each domain registered using a Task. All of this would be published to a very simple LiveView dashboard (to show LiveView’s potential, too).

I was also planning on using a document-store in GCE or AWS to store the registered domains (keeping the program very simple and sort of stateless, I’d like to try and run it in Kubernetes for testing).

I’m aware that this is a very simple task but I’d love any opinions or suggestions on this. I don’t want to over-engineer since this will be used only for a few websites, but I also don’t want to “brute force” everything, so to speak.

As a sidenote, I had the idea of doing this with LiveView but optimizing it for viewing through a terminal with cURL or similar programs. e.g. calling the service’s endpoint and start viewing a TUI dashbord. Would this be too crazy?

Thanks in advance.

Most Liked

mindok

mindok

There’s a good writeup (plus a library) for period tasks on @sasajuric 's blog - https://www.theerlangelist.com/article/periodic. This approach is very light on infrastructure. If you need full tracking, retry etc then the community seems to lean towards @sorentwo 's Oban project - https://github.com/sorentwo/oban - but that comes with more infrastructure (i.e database) to persist job status etc.

They should be able to give you some inspiration.

al2o3cr

al2o3cr

You mentioned you were tasked with the project at work; was there any discussion of build-vs-buy? It’s going to be challenging to show the benefits of Elixir when competing with off-the-shelf tools like Pingdom that do this exact job (plus alerting, dashboards, etc) for about 1 US dollar per month per site.

If there are requirements that aren’t served by the market - could be anything from regulatory requirements, to IP filtering on the target sites, to client confidentiality - then make sure your Elixir hype focuses on how it can help solve those unique problems.

mpope

mpope

A very simple way is using Process.send_after/4. A gen_server can just send itsself a repeating message.

cnck1387

cnck1387

There’s also https://uptimerobot.com/ which is free up to 50 sites and polls your site every 5 minutes. You can get alerts, a 30 day uptime history and response times on the free tier. I’ve been using them for years.

But if you do roll your own, database persistence seems reasonable so you can measure uptime over X time. That would likely mean storing the results in a DB, in which case Oban wouldn’t introduce additional complexity if you’re already using Postgres.

webuhu

webuhu

I did a similar project for monitoring, but never had time to finish it to be ready to open source it.

First of all often there aren’t such a thing as best practices in Elixir.
There could be more then one suitable good solution.
That’s one of the beautiful things in Elixir.

In my project for the scheduled “ping” requests it just used Task and TaskSupervisor.

The periodic task in its simplest form (in my opinion):

use Task, restart: :transient

# 60 s
@interval 60 * 1_000

def start_link(_) do
  Task.start_link(&process/0)
end

def process() do
  receive do
  after
    @interval ->
      # do your work here
      IO.inspect "ping"

      process()
  end
end

For storing the registered domains I just used a File which I read at application start.
About the TUI thing I can’t give any suggestions.

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list....
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
mgjohns61585
Could someone help me? I'm making my first elixir program, number guessing game. I can't figure out how to convert the user's guess from ...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement