Hi all, this post was incredibly helpful for me (THANK YOU!) and I wanted to share our final configuration, in case it helps anyone else. We’re using Google Kubernetes with libcluster. The Elixir apps are accessible by the web via a load balancer that use a google managed SSL certificate. They can access each other via that headless service.

env.sh.eex

#!/bin/sh
export RELEASE_DISTRIBUTION=name
export RELEASE_NODE=<%= @release.name %>@${POD_IP}

config/prod.exs

config :libcluster,
  topologies: [
    k8s: [
      strategy: Elixir.Cluster.Strategy.Kubernetes.DNS,
      config: [
        service: "myapp-service-headless",
        application_name: "my_app",
        polling_interval: 10_000
      ]
    ]
  ]

my_app.ex

  def start(_type, _args) do
    ...
    topologies = Application.get_env(:libcluster, :topologies) || []

    children = [
      {Cluster.Supervisor, [topologies, [name: MyApp.ClusterSupervisor]]},
      supervisor(MyApp.Repo, []),
      supervisor(MyAppWeb.Endpoint, []),
      ...
    ]

  opts = [strategy: :one_for_one, name: MyApp.Supervisor]

  Supervisor.start_link(children, opts)
end

kubernetes/all-together-now.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - image: gcr.io/my-gcp-project-123/my_app_otp:develop-HEAD
        name: myapp
        lifecycle:
          preStop:
            exec:
              command: ["./prod/rel/my_app/bin/my_app","stop"]
        ports:
        - containerPort: 4000
          protocol: TCP
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        envFrom:
        - configMapRef:
            name: staging-env-file
        command: ["./prod/rel/my_app/bin/my_app"]
        args: ["start"]
      terminationGracePeriodSeconds: 60   
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
  namespace: default
  annotations:
    cloud.google.com/app-protocols: '{"service-https-port":"HTTPS"}'
    cloud.google.com/neg: '{"ingress": true}' # Creates a NEG after an Ingress is created
spec:
  ports:
  - name: service-https
    port: 443
    protocol: TCP
    targetPort: 4000
  selector:
    app: myapp
  type: NodePort
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: staging-managed-ip
    ingress.gcp.kubernetes.io/pre-shared-cert: "staging"
    kubernetes.io/ingress.allow-http: "false"
spec:
  backend:
    serviceName: myapp-service
    servicePort: service-https
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service-headless
spec:
  ports:
    - port: 8000
  selector:
    app: myapp
  clusterIP: None

I’m doing a similar setup and this article helped me a lot https://engineering.dollarshaveclub.com/elixir-otp-applications-on-kubernetes-9944636b8609

repo: https://github.com/dollarshaveclub/ex_cluster

I haven’t done this in a long time but found an old app that used it.

config/prod.exs

config :libcluster,
    topologies: [
    k8s: [
      strategy: Cluster.Strategy.Kubernetes,
      config: [
        kubernetes_selector: "app=${KUBE_APP}",
        kubernetes_node_basename: "my_app"]]]

vm.args

-name my_app@${MY_POD_IP}
-setcookie <thecookie>

looking through my service files I don’t see anything interesting so I’m not sure they are relevant other than setting app name.

deployment

kind: Deployment
metadata:
  name: my_app-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my_app
    spec:

headless-service

apiVersion: v1
kind: Service
metadata:
  name: my_app-service-headless
  labels:
    app: my_app
spec:
  ports:
    - port: 8000
  selector:
    app: my_app
  clusterIP: None

service

apiVersion: v1
kind: Service
metadata:
  name: my_app-service
spec:
  ports:
    - port: 8080
      targetPort: 8000
      protocol: TCP
      name: http
  selector:
    app: my_app

Just realised I forgot the deployment one.

metadata:
  name: my_app-deployment
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my_app
    spec:
      imagePullSecrets:
        - name: regsecret
      containers:
        - name: my_app
          image: <my registry>
          ports:
            - containerPort: 8000
          args: ["foreground"]
          env:
            - name: HOST
              value: "app.myapp.com"

ingress (I think this has changed in newer versions though?) so probably not helpful.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my_app-appback-ingress
spec:
  rules:
  - host: appback.myapp.com
    http:
      paths:
      - path: /
        backend:
          serviceName: my_app-service
          servicePort: 8080

Let me know if you get it working. If not I’ll see if any other config seems relevant.