sergio

sergio

DropzoneJS direct uploads to Backblaze B2 in Phoenix Liveview

Wrote this guide on how to integrate DropzoneJS with Phoenix Liveview. Hope it helps someone out!

https://sergiotapia.com/dropzonejs-direct-uploads-to-backblaze-b2-in-phoenix-liveview


Woah read that title again, what a mouthful.

I’ve spent the last five evenings after work trying to get this to work. I’ve cobbled together blog posts, forums posts, github issues, hell I’ve spelunked chinese stackoverflow content cloner websites trying to figure this out.

It was a pain in the ass!

I’m going to teach you how to presign a Backblaze B2 upload url using the b2_get_upload_url API method. We’re then going to use that upload url to directly upload a file to your Backblaze B2 bucket from the browser.

Sounds simple right? Christ…

I hope this guide saves a collective thousands of hours out there. If this article helped you leave a comment please, I always enjoy reading those.


Create your Backblaze B2 bucket and set up CORS.

I’m not going to teach you how to create a bucket. Once you have your bucket, you need to use the Backblaze CLI to set up CORS for it. You cannot set up the right CORS rules in the Backblaze UI.

I’m using Linux, so I downloaded the Backblaze CLI and here’s how I run it.

Make sure you have the right environment variables set before running the b2-linux binary.

export B2_APPLICATION_KEY_ID=""
export B2_APPLICATION_KEY=""
export B2_APPLICATION_KEY_NAME="my-awesome-bucket-name"

Then update the CORS rules. Note that you cannot pass in a filename like foobar.json you must pass in the content itself. Lots of people trip with this one.

# I'm assuming you have the `b2-linux` binary in your folder...
./b2-linux update-bucket --corsRules '[
    {
        "corsRuleName": "downloadFromAnyOriginWithUpload",
        "allowedOrigins": [
            "*"
        ],
        "allowedHeaders": [
            "*"
        ],
        "allowedOperations": [
            "b2_download_file_by_id",
            "b2_download_file_by_name",
            "b2_upload_file",
            "b2_upload_part"
        ],
        "maxAgeSeconds": 3600
    }
]' my-awesome-bucket-name allPublic

Now your bucket is ready to receive XHR from the browser.

Presigning URLs.

We’re going to create a backblaze.ex module to do the presigning. I’m using req you can use whatever HTTP library you want.

defmodule MyApp.Backblaze do
  @b2_application_key System.get_env("B2_APPLICATION_KEY")
  @b2_application_key_id System.get_env("B2_APPLICATION_KEY_ID")
  @b2_application_key_name System.get_env("B2_APPLICATION_KEY_NAME")
  @b2_bucket_id System.get_env("B2_BUCKET_ID")

  def get_upload_url() do
    %{api_url: api_url, authorization_token: authorization_token} =
      get_api_url_and_authorization_token()

    request =
      Req.post!("#{api_url}/b2api/v2/b2_get_upload_url",
        headers: [{"authorization", "#{authorization_token}"}],
        json: %{bucketId: @b2_bucket_id}
      )

    request.body
  end

  def get_api_url_and_authorization_token() do
    auth_base_64 =
      "#{@b2_application_key_id}:#{@b2_application_key}"
      |> Base.encode64()

    response =
      Req.get!("https://api.backblazeb2.com/b2api/v2/b2_authorize_account",
        headers: [{"authorization", "Basic #{auth_base_64}"}]
      )

    api_url = response.body["apiUrl"]
    authorization_token = response.body["authorizationToken"]

    %{api_url: api_url, authorization_token: authorization_token}
  end
end

We also need an endpoint we can hit from the frontend.

scope "/api", MyAppWeb do
  pipe_through :api

  post "/presign-upload-url", PresignController, :presign
end

And the controller.

defmodule MyAppWeb.PresignController do
  use MyAppWeb, :controller
  alias MyApp.Backblaze

  def presign(conn, _params) do
    upload_url = Backblaze.get_upload_url()
    json(conn, %{upload_url: upload_url})
  end
end

Install DropzoneJS

Add dropzone to your package.json and npm i from inside of your /assets folder.

{
  "devDependencies": {
    "@tailwindcss/forms": "^0.5.2"
  },
  "dependencies": {
    "alpinejs": "^3.10.3",
    "dropzone": "^6.0.0-beta.2"
  }
}

Install the DropzoneJS css

Go to app.css and the import for Dropzone’s styles.

@import "tailwindcss/base";
@import "tailwindcss/components";
@import "tailwindcss/utilities";

 /* ADD THIS */
@import "dropzone/dist/dropzone.css";

Add DropzoneJS to your hooks.

In app.js you want to import a file we are going to create in the next step.

// We're going to create this file in the next step!
import dropzone from "./dropzone";

And add it to your hooks object.

let hooks = {};

// Add this...
hooks.Dropzone = dropzone;

let liveSocket = new LiveSocket("/live", Socket, {
  params: { _csrf_token: csrfToken },
  hooks: hooks,
  dom: {
    onBeforeElUpdated(from, to) {
      if (from._x_dataStack) {
        window.Alpine.clone(from, to);
      }
    },
  },
});

Prepare your DropzoneJS form

In whatever heex template you have, add a form.

<form
  class="dropzone dz-clickable"
  id="dropzone"
  phx-hook="Dropzone"
  phx-update="ignore"
  enctype="multipart/form-data"
>
</form>

Alright we’re done with all of the ceremony, now it’s time to actually upload. Are you ready? We’re almost there. Celebrate!

rambo.gif

Create your dropzone.js file

This should live in assets/js/dropzone.js.

import Dropzone from "dropzone";

export default {
  mounted() {
    let csrfToken = document
      .querySelector("meta[name='csrf-token']")
      .getAttribute("content");

    function initUpload(file) {
      return new Promise(function (resolve, reject) {
        fetch("/api/presign-upload-url", {
          headers: {
            "Content-Type": "application/json",
            "Accept": "application/json",
            "X-Requested-With": "XMLHttpRequest",
            "X-CSRF-Token": csrfToken
          },
          method: "post",
          credentials: "same-origin",
          body: JSON.stringify({
            key: file.name
          })
        })
          .then(function (response) {
            resolve(response.json());
          });
      });
    }


    let myDropzone = new Dropzone(this.el, {
      url: "#",
      method: "post",
      acceptedFiles: "image/*",
      autoProcessQueue: true,
      parallelUploads: 1,
      maxFilesize: 25, // Megabytes
      maxFiles: 5,
      uploadMultiple: true,
      transformFile: async function (file, done) {
        let initData = await initUpload(file);
        file.uploadUrl = initData.upload_url.uploadUrl;
        file.authorizationToken = initData.upload_url.authorizationToken;

        done(file);
      },
      init: function () {
        this.on("sending", function (file, xhr, formData) {
          xhr.open(myDropzone.options.method, file.uploadUrl);

          xhr.setRequestHeader("Content-Type", "b2/x-auto");
          xhr.setRequestHeader("Authorization", file.authorizationToken);
          // If you want to upload to a "folder" you just set it as part of the X-Bz-File-Name
          // for example you can set the username from a `window.username` variable you set.
          // xhr.setRequestHeader("X-Bz-File-Name", `yeyo/${Date.now()}-${encodeURI(file.name)}`);
          xhr.setRequestHeader("X-Bz-File-Name", `${Date.now()}-${encodeURI(file.name)}`);
          xhr.setRequestHeader("X-Bz-Content-Sha1", "do_not_verify");

          let _send = xhr.send
          xhr.send = function () {
            _send.call(xhr, file);
          }
        });

        this.on("success", function (file, response) {
          // The response has all the info you need to build the URL for your uploaded
          // file. Use it to set values on a hidden field for your `changeset` for example 
          console.log(response.fileId);
        });
      }
    });
  },
};

Guys… Dropzone is so vast and literally thousands of hours have gone into it by many different people but the documentation is very lacking. Herculean effort by the team and I owe them a lot of gratitude. It was painful for me to get this working. It’s very hard to discern what to do and many of the options contradict other options and make it behave strange.

Even the example this article has some weirdness: you can’t set parallelUploads: true otherwise only the last image actually gets uploaded. Weird right? But I’m happy with where this is today.

Hope this saved you at least an hour. If this didn’t work for you, please leave a comment and I’ll try to help out.

As next steps, throw up an imgproxy in front of your bucket for on the fly transformations. Then… throw up Bunny CDN in front of that for speed and lower your costs even more.

rambo.gif

Now get to uploading and save big bucks by using Backblaze B2.

Most Liked

mcrumm

mcrumm

Phoenix Core Team

Congrats on getting to the end on file uploads! :smiley: It can be a lot of work, which is why we designed LiveView uploads to remove as much complexity as possible–

  • allow_upload/3 enforces simple validations for accepted file types, max files/sizes, auto-upload, multiple uploads, etc. For direct-to-cloud uploads, the :external option is where you define a callback to generate a presigned URL– no additional controller necessary, and parallel uploads work out-of-the-box. :slight_smile:
  • live_file_input/2 can be wrapped in a container with a phx-drop-target attribute to handle drag and drop automatically.

I think the most custom code you need on the client-side is a custom external uploader. The guides are currently lacking an example of how to write your own uploader, so I linked to the S3 example which I think is closest to your needs. For clarity, the following covers a good portion of what you would need:

In your LiveView module:

def mount(_params, _session, socket) do
  {:ok,
   socket
   |> allow_upload(:files, accept: "image/*", max_entries: 5, max_file_size: 250000000, external: &presign_upload/2)}
end

defp presign_upload(entry, socket) do
  %{"uploadUrl" => url, "authorizationToken" => token} = MyApp.Backblaze.get_upload_url()

  {:ok, %{uploader: "Backblaze", url: url, token: token}, socket}
end

# phx-change: validate is a no-op but it is required for `live_file_input` validation.
def handle_event("validate", _, socket) do
  {:noreply, socket}
end

# phx-submit: save consumes the completed entries.
def handle_event("save", _, socket) do
  _results = 
    consume_uploaded_entries(socket, :files, fn %{} = _info, _entry ->
      # return {:ok, term()} to consume (complete) the entry.
      {:ok, nil}
    end)

  {:noreply, socket}
end

def render(assigns) do
  ~H"""
    <div class="container" phx-drop-target={@uploads.files.ref}>
      ...
      <form phx-change="validate">
        <%= live_file_input @uploads.files %>
        <button type="submit">Upload</button>
      </form>
    </div>
  """
end

In your Javascript, for instance in app.js:

let Uploaders = {}

Uploaders.Backblaze = function(entries, onViewError){
  entries.forEach(entry => {
    let { file, meta: { url, token } } = entry

    let xhr = new XMLHttpRequest()
    xhr.open("POST", url);

    xhr.setRequestHeader("Content-Type", "b2/x-auto");
    xhr.setRequestHeader("Authorization", token);
    // If you want to upload to a "folder" you just set it as part of the X-Bz-File-Name
    // for example you can set the username from a `window.username` variable you set.
    // xhr.setRequestHeader("X-Bz-File-Name", `yeyo/${Date.now()}-${encodeURI(file.name)}`);
    xhr.setRequestHeader("X-Bz-File-Name", `${Date.now()}-${encodeURI(file.name)}`);
    xhr.setRequestHeader("X-Bz-Content-Sha1", "do_not_verify");

    onViewError(() => xhr.abort())
    xhr.onload = () => xhr.status === 204 ? entry.progress(100) : entry.error()
    xhr.onerror = () => entry.error()
    xhr.upload.addEventListener("progress", (event) => {
      if(event.lengthComputable){
        let percent = Math.round((event.loaded / event.total) * 100)
        if(percent < 100){ entry.progress(percent) }
      }
    })

    let _send = xhr.send
    xhr.send = function () {
      _send.call(xhr, file);
    }
  })
}

let liveSocket = new LiveSocket("/live", Socket, {
  uploaders: Uploaders,
  params: {_csrf_token: csrfToken}
})

Let me know what you think of this approach, and if you have any questions I will do my best to answer them :slight_smile: Thanks for reading!

Where Next?

Popular in Guides/Tuts Top

voltone
The EEF’s Security WG has released the first public draft of the Secure Coding and Deployment Hardening Guidelines for BEAM languages. “...
New
kuon
I have a page with a large state that is loaded from DB, let’s call that “data”. I have a root live view that load “data” on mount and r...
New
tfwright
I thought I’d share a small project I’m working on to gain some familiarty with LiveView in a Phoenix app. Github Repo Deployment It’s...
New
anuragg
Hi everyone! I’m the founder of Render, a new cloud provider with native support for Elixir. When we launched Elixir support the most po...
New
hauleth
Some time ago someone suggested me to write article about how I have configured my Vim to work with Elixir, now there it is: https://med...
New
bentanweihao
I wrote a thing: http://engineering.pivotal.io/post/how-to-set-up-an-elixir-cluster-on-amazon-ec2/ Hope this is helpful!
New
cheerfulstoic
I spent quite a bit of time trying to find a good configuration to build / test my Elixir app in CircleCI and then push an image to Docke...
New
dgamidov
Hi folks, Just a short instruction. Maybe it will help somebody. https://v4-alpha.getbootstrap.com/getting-started Install boostrap...
New
1player
A question I had when first learning contexts and Ecto was how to expand the default context API to support more flexible queries. Usuall...
New
nietaki
Just a quick heads up: There seems to be a bug in Erlang/OTP 21.3, which can cause some errors when making http requests. If you’re using...
New

Other popular topics Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement