jxxcarlson

jxxcarlson

Elixir + files on heroku

I have a system where a tar file is received by an Elixir/Phoenix app, downloaded and an extracted, yielding a LaTeX file and a directory of image files. pdflatex is run twice and the location of the resulting pdf file is shipped back to the client. The client can then request the file. All this works on my laptop – but not on Heroku. I do get a message from the server indicating that extraction and pdflatex have succeeded. (See code at end). Also, the message below tells me that the tar achive has been saved.

However, when the client requests the file, it is not found.

I know the Heroku storage is ephemeral. However, for the short time I need these files, should generally be ok.

HEROKU LOG:

2019-01-12T22:27:44.217500+00:00 app[web.1]: params for 'process': %{"filename" => "bras_and_kets"}
2019-01-12T22:27:44.217507+00:00 app[web.1]: 22:27:44.212 request_id=bfb4cecb-7738-44f9-8838-1b27f968898c [info] POST /api/print/pdf/bras_and_kets
2019-01-12T22:27:44.217509+00:00 app[web.1]: BODY: <<98, 114, 97, 115, 95, 97, 110, 100, 95, 107, 101, 116, 115, 46, 116, 101, 120,
2019-01-12T22:27:44.217511+00:00 app[web.1]: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2019-01-12T22:27:44.217513+00:00 app[web.1]: 0, 0, 0, 0, 0, 0, 0, ...>>
2019-01-12T22:27:44.217730+00:00 app[web.1]: PATH: printfiles/bras_and_kets/bras_and_kets.tar
2019-01-12T22:27:44.237525+00:00 app[web.1]: XX, FILE EXISTS: printfiles/bras_and_kets/bras_and_kets.tar
2019-01-12T22:27:44.338354+00:00 app[web.1]: 22:27:44.337 request_id=bfb4cecb-7738-44f9-8838-1b27f968898c [info] Sent 200 in 125ms
2019-01-12T22:28:14.005012+00:00 heroku[router]: at=info method=GET path="/print/pdf/bras_and_kets" host=nshost.herokuapp.com request_id=4c2b7a3a-6acb-431b-9d73-6571833ff720 fwd="24.92.138.170" dyno=web.1 connect=1ms service=38ms status=200 bytes=611 protocol=https
2019-01-12T22:28:14.005494+00:00 app[web.1]: 22:28:14.001 request_id=4c2b7a3a-6acb-431b-9d73-6571833ff720 [info] GET /print/pdf/bras_and_kets
2019-01-12T22:28:14.006072+00:00 app[web.1]: 22:28:14.005 request_id=4c2b7a3a-6acb-431b-9d73-6571833ff720 [info] Sent 200 in 4ms

CONTROLLER FUNCTION 1: handle POST request sending tar archive

def process(conn, params) do

    IO.inspect params, label: "params for 'process'"
    {:ok, body, conn} = Plug.Conn.read_body(conn, length: 3_000_000)
    IO.inspect body, label: "BODY"

    bare_filename = params["filename"]
    tarfile = "#{bare_filename}.tar"
    texfile = params["filename"] <> ".tex"
    prefix = "printfiles/#{params["filename"]}"
    {:ok, cwd} = File.cwd
    File.mkdir_p prefix
    tar_path = "#{prefix}/#{tarfile}"
    IO.puts "PATH: " <> tar_path
    {:ok, file} = File.open tar_path, [:write]
    IO.binwrite file, body
    File.close file

    case File.read(tar_path) do
      {:ok, body} -> IO.puts "XX, FILE EXISTS: #{tar_path}"
      {:error, reason} -> IO.puts "XX,  NO SUCH FILE: #{tar_path}"
    end

    # System.cmd("tar", ["xvf", path])
    System.cmd("tar", ["-xf", tar_path, "-C", prefix ])
    File.cd prefix
    System.cmd("pdflatex", ["-interaction=nonstopmode", texfile])
    System.cmd("pdflatex", ["-interaction=nonstopmode", texfile])
    File.cd cwd

    conn |> render("pdf.json", url: bare_filename)
  end

CONTROLLER FUNCTION 2: get the pdf file:

  def display_pdf_file(conn, %{"filename" => filename}) do
    path = "printfiles/#{filename}/#{filename}.pdf"
    case File.read(path) do
      {:ok, body} -> Plug.Conn.send_file(conn, 200, path)
      {:error, reason} -> conn |> render("pdf_error.html", path: "Sorry, couldn't find the PDF file.")
    end
  end

Also: I see no directory printfiles at the root level of the Heroku directory, where I expect it to be.

Most Liked

NobbZ

NobbZ

Have you checked if the latex run is done without an error?

{_, 0} = System.cmd("pdflatex", args, stderr_to_stdout: true)

Perhaps even use the first element of the tuple to collect combined stdout/stderr and check what the error was (if there was one).


edit

Is pdflatex even installed on heroku?

jxxcarlson

jxxcarlson

Using your method to capture pdflatex errors, I’ve found the source of the problem. My .tex files use some .sty files that are not in the TeXLIve distribution defined by the buildpack. Trying to figure out how to resolve these.

Thanks so much for your help!

frigidcode

frigidcode

Previously I’ve created *.deb files and installed them with the apt-buildpack to supplement the install. Might help you in this situation.

frigidcode

frigidcode

This is what I used when I started making packages for Heroku.

I then hosted them on S3, and added the URL to Aptfile.

jxxcarlson

jxxcarlson

Thankyou! Looks very good … will try to set uo tomorrow.

Where Next?

Popular in Questions Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
Werner
Hi, I’m using Ubuntu 18.04 and after updating to OTP-24.0 yesterday i have this warning when I run “mix local.hex”: 14:57:30.512 [warn] ...
New
polypush135
As many of you may have realized by now (sorry for all the posts here) I’ve been working on a db problem where I’m trying to aggregate a ...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New

Other popular topics Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
KronicDeth
Elixir plugin for JetBrain’s IntelliJ Platform (including Rubymine) This is a plugin that adds support for Elixir to JetBrains IntelliJ...
289 35421 110
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement