bigfish1

bigfish1

Unable to recreate file after downloading it in chunks

I am trying to download files from Azure Blob Storage to process after they are uploaded by my users through a separate client app. For small files I have no issue downloading the data and writing it to a file, however larger ones timeout so I am using the http range header to download the file in smaller parts then write it to a file. My issue is that after downloading all the data I get an error (ErlangError) Erlang error: :no_translation:

If I download a small file that the entire thing fits inside the range I provide the same code works perfectly. This leads me to believe I may be doing something wrong when I combine the bitstrings.

Here is the code

      ranges = blob_ranges(get_blob_length!(blob_name, player_id)) # Ranges to be used for chunking files
      {last_byte, data} = Enum.reduce(ranges, {0, ""}, fn end_byte, acc  ->
        case get_blob!(blob_name, player_id, [timeout: 120], [{"x-ms-range", "bytes=#{elem(acc, 0)}-#{end_byte}"}]) do
          blob -> 
            {end_byte+1, elem(acc, 1) <> blob}
          {:error, err} -> 
              IO.puts("Request failed: #{Exception.message(err)}")
        end
      end)
      create_temp_file(blob_name, data)

Most Liked

derek-zhou

derek-zhou

This will not work. You chunk may straddle across a UTF-8 multi-byte sequence. You need to concatenate the binary raw.

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
kostonstyle
Hi all I want to have a unix time, from the current time plus 1 hour. DateTime.now + 1 hour How to get it in elixir? Thanks
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New

We're in Beta

About us Mission Statement