asiniy

asiniy

Download - the easiest way to download files from the internet

Hey there!

I wrote a download elixir package which does exactly what its name about - an easy way to download files.

I saw solutions about how to do it with HTTPoison but these solutions have disadvantages:

  • The whole response is loading to the RAM, and only after then is going to the File.write/1. You have to choose async HTTPoison request to escape high memory consumption.
  • There is no restriction of the file size to download. If you handle, saying, user input and then you’re trying to download any file user provided, your server can go down for downloading a file which has 1TB size.

download depends on HTTPoison and successfully sidestep these issues.

Enjoy :wink:

Most Liked

NobbZ

NobbZ

This project is about downloading a single resource from the specified URL. It does not any magic to determine what the resource actually is. It gets saved as is and then you get path of the saved file.

This is in contrast to using HTTPoison without thinking about it and loading the full response into RAM.

One can get the same behaviour with HTTPosion directly though using async requests, but this package makes it a bit easier to process those.

gon782

gon782

Important to note is that with, for example, youtube-dl you also get support for downloading from Twitch and vimeo.

Eiji

Eiji

@asiniy: Nice, but I have a few ideas:

  1. Support for POST requests, because some pages requires it
  2. Support for parameters (GET) and data (POST), because it’s easier to read code (parameters) + some pages requires POST with data
  3. Support for finders, for example: Download.from(url, finder: Download.Finders.Video)
  4. Support for Traditional units
NobbZ

NobbZ

I do not like that page you linked @Eiji, it seems to confuse metric and binary prefixes.

Even gives megabyte as an example for binary and mebibyte as example while on the linked pages about binary and metric prefixes they are the other way round.

So I am not sure what you exactly mean by “traditional units”?

outlog

outlog

I would think you just downloaded the html file.

You probably need to use something like https://github.com/rg3/youtube-dl/ to get to the video file - you might want to use Porcelain to interface it https://github.com/alco/porcelain

Where Next?

Popular in Libraries Top

OvermindDL1
Been making an MLElixir thing (not released yet…) for fun in spare time in the past day. I’m just trying to see how much I can get an ML...
132 13487 106
New
achempion
Hi, I would like to tell about my initiative to further maintain and develop Waffle project which is the fork of Arc library. The progre...
New
mhanberg
I just released the first version of Temple: an HTML DSL for Elixir and Phoenix! You can read this blog post or the docs for more info...
New
dbern
I’m excited to announce that TaxJar has developed and open-sourced DateTimeParser. We developed it because we found a need to parse user ...
New
riverrun
I’ve just released version 3 of Comeonin, a password hashing library. The following small changes have been made: changes to the NIF c...
New
oltarasenko
Dear Elixir community, After a year of development, bug fixes, and improvements, we are proudly ready to share the release of Crawly 0.1...
New
Crowdhailer
I have been updating a library that allows you to pipe between functions that use the erlang result tuple convention. Assuming you have...
New
martinthenth
Hello everybody :wave: Recently, some of my colleagues talked about database ids and uuids and their problems, and I remembered the pain...
New
engineeringdept
I’ve just released the first version of Snap, an Elasticsearch client. It borrows ideas about application structure and process managemen...
New
KallDrexx
For a good number of months I've been working on creating a very basic RTMP live video streaming server. Now that I have a very, very ba...
New

Other popular topics Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New
_russellb
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New

Sub Categories:

We're in Beta

About us Mission Statement