Zsolt

Zsolt

Using FLAME to run code in ephemeral machines

I’m trying to get some code to run in Flyio Flame machines.

I have a phoenix app (although, I think the question applies more generally to Elixir and Flame).

There is a live file upload module that takes a few files, and using an image library resizes, converts and saves them to disk. The path to the images is saved in a Postgres db. Very simplistic, this is a learning tool, not a production app.

Resizing and saving the files is not CPU intensive, but it can use up a lot of memory, so it would help to do each operation on a new machine. (I’m aware other kinds optimizations are possible, but the purpose of this is to try Flame).

This blog post (Rethinking Serverless with FLAME · The Fly Blog) seems to state that it’s enough to wrap a chunk of code in a FLAME.call with a FLAME.Pool and those chunks should run on separate machines.

That’s it! FLAME.call accepts the name of a runner pool, and a function. It then finds or boots a new copy of our entire application and runs the function there.

I have this implementation and I’m not seeing and Flame machines start up. Am I doing something wrong? Did I misunderstand the post, and do I in fact need the entire GenServer and FLAME.place_child functionality?

The FLAME.Pool in /lib/phoenix_albums/application.ex

  @impl true
  def start(_type, _args) do
    flame_parent = FLAME.Parent.get()

    children =
      [
        PhoenixAlbumsWeb.Telemetry,
        PhoenixAlbums.Repo,
        {DNSCluster, query: Application.get_env(:phoenix_albums, :dns_cluster_query) || :ignore},
        {Phoenix.PubSub, name: PhoenixAlbums.PubSub},
        # Start the Finch HTTP client for sending emails
        {Finch, name: PhoenixAlbums.Finch},
        # Start a worker by calling: PhoenixAlbums.Worker.start_link(arg)
        # {PhoenixAlbums.Worker, arg},
        # Start to serve requests, typically the last entry
        {FLAME.Pool,
         name: PhoenixAlbums.ImageProcessor, min: 0, max: 10, max_concurrency: 1, log: :debug},
        !flame_parent && PhoenixAlbumsWeb.Endpoint
      ]
      |> Enum.filter(& &1)

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: PhoenixAlbums.Supervisor]
    Supervisor.start_link(children, opts)
  end

My upload handler in the live fodler:

 @impl Phoenix.LiveView
  def handle_event("save", params, socket) do
    unless File.exists?(@folder) do
      File.mkdir!(@folder)
    end

    uploaded_files =
      consume_uploaded_entries(socket, :avatar, fn %{path: path}, entry ->
   
        uuid = UUID.generate()
        image_folder = "#{@folder}/#{uuid}"

        unless File.exists?(image_folder) do
          File.mkdir!(image_folder)
        end

        FLAME.call(
          PhoenixAlbums.ImageProcessor,
          fn -> resize_image(:thumbnail, path, 150, image_folder) end
        )

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          resize_image(:medium, path, 800, image_folder)
        end)

        FLAME.call(PhoenixAlbums.ImageProcessor, fn ->
          save_original(path, image_folder)
        end)

        image = Map.merge(params, %{"user_id" => socket.assigns.user_id, "url" => image_folder})

        case Album.create_image(image) do
          {:ok, image} ->
            socket
            |> put_flash(:info, "Image created successfully.")

            {:ok, ~p"/images/#{image}"}
        end
      end)

    {:noreply, update(socket, :uploaded_files, &(&1 ++ uploaded_files))}
  end

Here I just wrapped resize_image and save_original functions, with the expectation, that these would be run on separate machines. The code runs on fly just the same as without the wrappers, and I’m not sure why.

Marked As Solved

chrismccord

chrismccord

Creator of Phoenix

Have you configured the FlyBackend? You can either pass the backend to the pool options, or per the FlyBackend docs:

The only required configuration is telling FLAME to use the
FLAME.FlyBackend by default and the :token which is your Fly.io API
token. These can be set via application configuration in your config/runtime.exs
withing a :prod block:

  if config_env() == :prod do
    config :flame, :backend, FLAME.FlyBackend
    config :flame, FLAME.FlyBackend, token: System.fetch_env!("FLY_API_TOKEN")
    ...
  end

Where Next?

Popular in Questions Top

pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
Phillipp
Hey, I have a NanoPi-M3 and try to install Elixir on their Ubuntu image. I followed the Raspberry Pi installation instructions from the ...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
chrismccord
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

We're in Beta

About us Mission Statement