m_tofi

m_tofi

How do I know when Finch.stream has finished fetching data?

I am using Finch.stream to fetch a large CSV file from a server, but neither the ‘done’ state nor the ‘error’ state is being triggered when I use "{:halt, {:error, reason}}". I need help figuring out how to determine when the stream has finished fetching data.

Finch.build(:get, url, @headers)
    |> Finch.stream(
      Pdex.Finch,
      {"", {nil, nil}},
      fn
        {:status, status}, acc ->
          IO.inspect(status)
          {:cont, acc}

        {:headers, headers}, acc ->
          IO.inspect(headers)
          {:cont, acc}

        {:data, data}, acc ->
          {:cont, {:cont, {rest, _state} = order}} = acc

          chunk = rest <> data

          # {:halt, {:error, "force not work"}}

          {:cont, {chunk, state}}
       
        # never Geting Call
        {:error, reason}, acc ->
          IO.inspect(acc, label: "Error")
          {:halt, {:error, reason}}

        # never geting Call
        {:done, _}, acc ->
          IO.inspect(acc, label: "Done")
          {:ok, :done}
      end,
      receive_timeout: 30000
    )
  end

Marked As Solved

akash-akya

akash-akya

Finch.stream returns the acc on successful completion. You can use that determine if the stream has finished.

Finch.stream(Finch.build(:get, url), MyFinch, nil, fn
  {:status, status}, nil ->
    IO.inspect(status: status)
    0

  {:headers, headers}, length ->
    IO.inspect(headers: headers)
    length

  {:data, data}, length ->
    IO.inspect(data: IO.iodata_length(data))
    length + IO.iodata_length(data)
end)
|> case do
  {:ok, length} ->
    # streaming complete, handle `:done` state
    IO.inspect(total_length: length)
    
  {:error, error} -> 
    raise error
end

Also Liked

zachallaun

zachallaun

I think you’re looking for stream_while instead of stream, documented here. The stream function ignores the value returned from the given callback, whereas stream_while will look for the :cont/:halt tuples.

Where Next?

Popular in Questions Top

yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Exadra37
Sometimes I want to check if the input into a function is not a blank string. My first approach: defmodule Example do def do_stuff(s...
New
hpopp
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

shahryarjb
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement