elsatch

elsatch

File management sample codes?

Hi everyone!

My name is César and I’m learning how to use Elixir. My goal is to create some kind of web crawler to locate historical data. Anyway, right now, I am into a more mundane task.

I’m trying to create some programs in Elixir than help me sort my files. For example, I’d like to sort my files by extension or order my photos by modification date. I thought that it would be an easy task, but there are not so many examples of how to traverse directories and move files. I’ve checked on Exercism and several online books, but I am always taken back to the reference manual.

I’ve seen the Files.rename/2 information, but given my current Elixir level, I’m not sure about how to combine it with safeguards so the results are not disastrous. (For example, I might have several source files called IMG_0001.jpg and I don’t want to have them silently overwritten).

Do you know any source that might have this kind of file management examples? Any cookbook that covers this? Most resources flow like Install |> Strings and binaries |> Flow control |> OTP BeamVM, skipping the file chapter :slight_smile:

Thanks for your support!

Most Liked

Aetherus

Aetherus

defmodule sort_files_by_extension do
  def sort_files do
    File.ls!("./test")
    |> Enum.map(fn filename -> 
         extname = Path.extname(filename)  #=> ".jpg"
         basename = Path.basename(filename, extname)  #=> "IMG_XXXX"
         {basename, extname, filename}
       end)
    |> Enum.map(&make_extension_dir/1)
    |> Enum.map(&dedup/1)
    |> Enum.each(&move_extension_directory/1)

  defp make_extension_dir({_, "." <> extension, _} = arg) do
    File.mkdir_p!("./#{extension}")
    arg  # just return the argument for other `Enum.map`
  end

  defp dedup(arg, suffix \\ 0)

  defp dedup({basename, "." <> extension, filename} = arg, 0) do
    if File.exists?("#{extension}/#{filename}") do
      dedup(arg, 1)
    else
      arg
    end  
  end

  defp dedup({basename, "." <> extension = extname, filename} = arg, suffix) do
    if File.exists?("#{extension}/#{basename}_#{suffix}.#{extension}") do
      dedup(arg, suffix + 1)
    else
      {"#{basename}_#{suffix}", extname, filename}
    end 
  end

  defp move_to_extension_directory({basename, "." <> extension, filename}) do
    File.rename!("./test/#{filename}", "./#{extension}/#{basename}.#{extension}")
  end
end

A few suggestions:

  1. Enum.each is for pure side effects (e.g. pure file system operations). It’s not chainable. Use Enum.map instead.
  2. You can wrap all the information in a tuple, and pattern match on it in the function parameters.
  3. You can trust the file system to do the right thing (e.g. File.mkdir_p!)
  4. Hail recursion!
dimitarvp

dimitarvp

This might not be the answer you are looking for – but I’d reach for sqlite3. It also can be up to 35% faster than raw file access.

Trouble is, current state of the art of sqlite3 in Elixir does not support Ecto 3 (latest version of the de facto DB persistence library); only version 2 which is now quite old. I am working on bringing Ecto 3 to sqlite3 but it definitely is not going to be ready tomorrow.

Using sqlite3 will rid you of the potential file overwriting problem as well since there the ID is… you know, the database ID, not the filename itself. What’s more, an embedded DB like sqlite3 gives you the ability to sort and filter out of the box.


If that doesn’t sound tempting to you then I’d be happy to help you exactly with the File / IO API. However, you should have in mind that functions like stat can have differing behaviours between different OS-es. Which is all the more reason to opt for a database.

Any particular scenario you would like help with?

If you would like something that basically manages uploads to your app then Waffle might be exactly what you are looking for (it can put all stored files into Amazon’s S3 or in your local filesystem).

OvermindDL1

OvermindDL1

I’d probably do this:

  def sort_files(path) do
    files_by_dir =
      File.ls!(path)
      |> Enum.reject(&File.dir?/1)
      |> Enum.group_by(&Path.extname/1)

    Enum.each(files_by_dir, fn {"." <> ext, files} ->
      extpath = Path.join(path, ext)
      File.mkdir_p(extpath)

      Enum.each(files, fn file ->
        filepath = Path.join(extpath, file)
        # Keep only a max of 9, could easily make this unbounded though, but eh useful feature to add
        Enum.each(9..2, &File.rename("#{filepath}_#{&1-1}", "#{filepath}_#{&1}"))
        File.rename(filepath, "#{filepath}_1")
        File.rename!(Path.join(path, file), filepath)
      end)
    end)
  end

Could use some error reporting, but eh.

Where Next?

Popular in Questions Top

dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
logicmason
Hi there, I'm working through my first release with elixir/phoenix. I've built a release with distillery and found that it crashes when I...
New
sacepums
Hey guys. I'm new to elixir and im really stocked about it. But I ran into a bit of problem - I need to convert a date sting, for examp...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New

Other popular topics Top

peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
johnnyicon
Hi all, I've just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I'm trying to use Postg...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

We're in Beta

About us Mission Statement