x4K3PqNH

x4K3PqNH

Read file (csv) uploaded via POST in Plug.Router by only using Plug libraries, no Phoenix awailable

The closest I could get to read the file was seeing it in the body

defmodule Foo.Bar do
  use Plug.Router

  import Plug.Conn
  require Logger

  plug Plug.Logger, log: :debug

  plug :match
  plug :dispatch
  plug(Plug.Parsers, parsers: [:urlencoded, {:multipart, length: 10_000_000}])

  post "/foo/bar/import" do
    Logger.error(inspect conn.params["file"])
    Logger.error(inspect(Plug.Conn.read_body(conn), label: "body"))
  end
end

And here is the output of Logger:

2023-02-20 20:57:52.722 [error] nil (module=Foo.Bar line=14 )
2023-02-20 20:57:52.740 [error] {:ok, "-----------------------------2239044772062058215875569635\r\nContent-Disposition: form-data; name=\"file\"; filename=\"example.csv\"\r\nContent-Type: text/csv\r\n\r\nh1;h2;h3\nr1c1;r1c2;r1c3\nr2c1;r2c2;r2c3\nr3c1;r3c2;r3c3\n\r\n-----------------------------2239044772062058215875569635--\r\n", %Plug.Conn{adapter: {Plug.Cowboy.Conn, :...}, assigns: %{}, body_params: %Plug.Conn.Unfetched{aspect: :body_params}, cookies: %Plug.Conn.Unfetched{aspect: :cookies}, halted: false, host: "192.168.11.51", method: "POST", owner: #PID<0.812.0>, params: %{}, path_info: ["foo", "bar", "import"], path_params: %{}, port: 80, private: %{before_send: [#Function<1.3486200/1 in Plug.Logger.call/2>], plug_route: {"/foo/bar/import", #Function<1.57698496/2 in Foo.Bar.do_match/4>}}, query_params: %Plug.Conn.Unfetched{aspect: :query_params}, query_string: "", remote_ip: {127, 0, 0, 1}, req_cookies: %Plug.Conn.Unfetched{aspect: :cookies}, req_headers: [{"accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"}, {"accept-encoding", "gzip, deflate, br"}, {"accept-language", "en-US,en;q=0.5"}, {"authorization", "[[content removed]]"}, {"cache-control", "no-cache"}, {"content-length", "271"}, {"content-type", "multipart/form-data; boundary=---------------------------2239044772062058215875569635"}, {"cookie", "session=1668075015;"}, {"dnt", "1"}, {"host", "192.168.11.51"}, {"origin", "https://192.168.11.51"}, {"pragma", "no-cache"}, {"referer", "https://192.168.11.51/"}, {"sec-fetch-dest", "iframe"}, {"sec-fetch-mode", "navigate"}, {"sec-fetch-site", "same-origin"}, {"sec-gpc", "1"}, {"te", "trailers"}, {"upgrade-insecure-requests", "1"}, {"user-agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/109.0"}, {"x-forwarded-for", "10.0.8.12"}, {"x-forwarded-host", "192.168.11.51"}, {"x-forwarded-port", "443"}, {"x-forwarded-proto", "https"}, {"x-forwarded-server", "root-uld-trk5"}, {"x-real-ip", "10.0.8.12"}], request_path: "/foo/bar/import", resp_body: nil, resp_cookies: %{}, resp_headers: [{"cache-control", "max-age=0, private, must-revalidate"}], scheme: :http, script_name: [], secret_key_base: nil, state: :unset, status: nil}} (module=Foo.Bar line=16 )

Unfortunately getting from here to actually read the file part of the body got me stuck, found no valid example, tried many things I found online that seem to be deprecated as no longer available in Plug 1.14, like get_upload/2, Plug.Upload.parse_multipart/2, get_multipart_params/2, Plug.Upload.get_params/1, Plug.Upload.get/2.

Tried to get_req_header and if it’s “multipart/form-data” do Plug.Multipart.parse or Plug.Parsers.MULTIPART.parse/5 that’s deprecated I guess or I failed to understand how to use, then trying with a recursive with Plug.Parsers.MULTIPART.init([]) and Plug.Parsers.MULTIPART.parse/5

I am definitely miss something here, anyone has a valid example E2E on how can I read the file in the Plug above by only using plug, where the files is clearly visible in the body that’s “multipart/form-data” (P.S. I can’t control the requester to change the content-type).

Thank you

Marked As Solved

al2o3cr

al2o3cr

Apologies - I misread RFC2046 and completely missed the part where the lines separating the parts are -- followed by the boundary! The request is correctly formed.

The issue is simpler: Plug.Parser isn’t setting params because it isn’t being called, because it’s listed after plug :dispatch.

Also Liked

x4K3PqNH

x4K3PqNH

I was not sure where to post more code, so I made a minimum project that reveals this, the code is in api.ex where you can also see lots of things commented out from things I tried.
I also attached a har file as exported from the browser when doing the upload, and the body of the request as I see in the logs if I read it via Plug.Conn.read_body(conn), label: "body")

If there are other suggestions on where to post the code to be easier please let me know.
Thank you

Where Next?

Popular in Questions Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
clayschick
I'm trying to create a simple query to select distinct values from a column. If I only use select and distinct I get back a list of uniqu...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement