mruoss

Flame_k8s_backend - a FLAME Backend for Kubernetes

I’m working on the FLAME Backend for Kubernetes.

In my current PR, I’d like to allow for better control over the runner pod manifest. The current approach basically offers 2 ways of controlling the runner pod manifest.

In the simpler case you can just define env vars and resource requests/limits for the runner pods. The FLAME backend then creates the runner pod with these values set.

If you need more advanced features like pod affinity (e.g. running on GPU nodes), volumes etc, you can implement a callback in which you build the runner pod manifest in your application and return it to the FLAME backend. The backend then adds soem required env variables, set/overwrite a few values like the pod name, container image, etc. and finally apply it to the cluster to create the runner pod.

Inputs anyone?

github.com/mruoss/flame_k8s_backend

Runner pod templates

mruoss:main ← mruoss:runner-pod-template

opened 07:10PM - 13 Dec 23 UTC

mruoss

+736 -113

## Module Docs of `FLAMEK8sBackend.RunnerPodTemplate` This module is respon…sible for generating the manifest for the runner pods. The manifest can be overridden using the `runner_pod_tpl` option on `FLAMEK8sBackend`. ### Simple Use Case By default, `resources` and `env` variables are copied from the parent pod. Using the `runner_pod_tpl` option on the `FLAMEK8sBackend`, you can add additional environment variables or set different `resources`. You would do this by setting the `runner_pod_tpl` to a struct of type `t:FLAMEK8sBackend.RunnerPodTemplate.t/0` as follows: ```ex # application.ex alias FLAMEK8sBackend.RunnerPodTemplate children = [ {FLAME.Pool, name: MyApp.SamplePool, backend: {FLAMEK8sBackend, runner_pod_tpl: %RunnerPodTemplate{ env: [%{"name" => "FOO", "value" => "bar"}], resources: %{ requests: %{"memory" => "256Mi", "cpu" => "100m"}, limimts: %{"memory" => "256Mi", "cpu" => "400m"} } } }, # other opts } ] # ... ``` ### Advanced Use Case - Passing a Callback Function In some cases you might need advanced control over the runner pod manifest. Maybe you want to set node affinity because you need your runners to run on nodes with GPUs or you need additional volumes etc. In this case, you can pass a callback via `runner_pod_tpl` option to the `FLAMEK8sBackend`. The callback has to be of type `t:FLAMEK8sBackend.RunnerPodTemplate.callback/0`. The callback will be called with the manifest of the parent pod which can be used to extract information. It should return a pod template as a map Define a callback, e.g. in a separate module: ```ex defmodule MyApp.FLAMERunnerPodTemplate do def runner_pod_manifest(parent_pod_manifest) do %{ "metadata" => %{ # namespace, labels, ownerReferences,... }, "spec" => %{ "containers" => [ %{ # container definition } ] } } end end ``` Register the backend: ```ex # application.ex # ... children = [ {FLAME.Pool, name: MyApp.SamplePool, backend: {FLAMEK8sBackend, runner_pod_tpl: &MyApp.FLAMERunnerPodTemplate.runner_pod_manifest/1}, # other opts } ] # ... ``` > #### Predefined Values {: .warning} > > Note that the following values are controlled by the backend and, if set by > your callback function, are going to be overwritten: > > * `apiVersion` and `Kind` of the resource (set to `v1/Pod`) > * The pod's and container's names (set to a combination of the parent pod's > name and a random id) > * The `restartPolicy` (set to `Never`) > * The container `image` (set to the image of the parent pod's app container) ### Options * `:omit_owner_reference` - Omit generating and appending the parent pod as `ownerReference` to the runner pod's metadata * `:app_container_name` - name of the container running this application. By default, the first container in the list of containers is used.

24 comments

#kubernetes #flame

49 2285 24

2024-08-13 19:41:57 UTC

Most Liked

entone

No feedback at this point, but just wanted to say thank you! I was hoping this would happen quickly!

Post #2

chrismccord

Creator of Phoenix

No worries at all – I didn’t take it that way Start times fluctuate for sure based on what docker image layers are cached locally or not. On fly, machine creates will favor a host with the current image cached if that host has available capacity, falling back to pulling from a region registry. Also one option for a faster FlyBackend would be to precreate the machines and then stop (or suspend) them. Then growing the flame pool is a machine start instead of create, which would bring starts down to 500ms. I assume the k8s orchestrator is doing something along these lines.

I have avoided this so far on the FlyBackend because it requires more orchestration and failsafes to avoid orphaning resources, and the cold boots are Fast Enough™ . That said, with the new machine suspend features we can technically have a machine back up and running sub 100ms. I still favor the cold creates for simplicity, but I would be open to an option like eager: true which precreates the fly machines if folks really want it.

Post #25

mruoss

In version 0.4.1 released yesterday I have removed Req - the last dependency besides Flame. Now FLAME can safely be used in Livebooks running on Kubernetes.

Post #18

chrismccord

Creator of Phoenix

http://rtt.fly.dev launches a flame every 10 min from each region:

Post #23

mruoss

I have implemented the workaround in flame_k8s_backend for now until this issue is fixed in one of the lower layers. GH issues are open on OTP and Mint. Thanks a lot @voltone for your help and let’s take this discussion to GitHub now.

And thanks @maennchen for raising this!

Post #16

Where Next?

View thread on forum (has 24 responses!)

kubernetes

flame

Home Libraries & Projects>Libraries

#kubernetes #flame

49 2286 24

Last post

Flame_k8s_backend - a FLAME Backend for Kubernetes

mruoss

Flame_k8s_backend - a FLAME Backend for Kubernetes

Most Liked

entone

chrismccord

mruoss

chrismccord

mruoss

Where Next?

Popular in Libraries

Telemetry - Dynamic dispatching library for metrics and instrumentations (v0.3.0 released!)

Solution: library to do pattern-matching with general ok/error types in case/with statements -- v1.0.1

Image - an image processing library based upon Vix

Comeonin password hashing library

IntelliJ Elixir - Elixir plugin for JetBrain's IntelliJ Platform

OpenApiSpex - OpenAPI / Swagger 3.0 for Plug APIs

PersistentEts - lightweight persistence over ets

PidFile - create and manage a PID file from the BEAM process

WebSockex - An Elixir WebSocket client

Elixir Media Libs

Other popular topics

** (RuntimeError) could not lookup Ecto repo Repo because it was not started or it does not exist

(Postgrex.Error) FATAL 28P01 (invalid_password) password authentication failed for user “postgres”

How to set up the Elixir SDK in Intellij IDEA with the intellij-elixir plugin?

Ecto query using like/ilike in query

Phoenix 1.6.0 LiveView + esbuild + Tailwind JIT + AlpineJS - A brief tutorial

Emacs - Elixir Setup Configuration Wiki

Web scraping tools

Removing spaces and characters from string

Write while loop equivalent in elixir

Ecto delete a record WITHOUT selecting first

Libraries & Projects>Libraries

Latest on Elixir Forum

Sponsor Spotlight

Our Sponsors

Categories:

Sub Categories:

Forums

Our Sponsors

We're in Beta