ConnorRigby

ConnorRigby

Nerves Core Team

Help with `:file` blocking GenServer process

I have a GenServer that reads/writes to a Linux Pipe or Fifo. I need to do:

{:ok, fifo} = :file.open('/path/to/fifo', [:read, :write, :binary])
{:ok, <<protocol_pattern_match>>} = :file.read(fifo, protocol_size)

Except :file.read/2 blocks the entire calling process until it is complete similar to read() in C. This is fine because i should be able to just do task = Task.async(:file, :read, [protocol_size]) and get the result in
handle_info/2. Maybe i misunderstood the docs, but that doesn’t seem to be working for me. I’m expecting to get handle_info({ref, {:ok, <<protocol_pattern_match>>}, %{task: %{ref: ref}}) but that doesn’t seem to happen.

My other issue is that Task.shutdown/2 or :file.close/1 do not seem to be working as expected.
the docs for Task.shutdown/2 say when a calling process exits, the task should exit, but even using
Task.shutdown(state.task, :brutal_kill) doesn’t allow me to call :file.close(state.fifo) in terminate/2. (It just blocks forever)

anyway here’s the entire GenServer implementation:

defmodule PipeWorker do
  @moduledoc """
  Proxy for IO operations.
  """
  use GenServer
  require Logger

  def start_link(pipe_name) do
    GenServer.start_link(__MODULE__, pipe_name)
  end

  def close(pipe) do
    GenServer.stop(pipe, :normal)
  end

  def read(pipe, size) do
    GenServer.call(pipe, {:read, [size]}, :infinity)
  end

  def write(pipe, packet) do
    GenServer.call(pipe, {:write, [packet]}, :infinity)
  end

  def init(pipe_name) do
    with {_, 0} <- System.cmd("mkfifo", [pipe_name]),
         {:ok, pipe} <- :file.open(to_charlist(pipe_name), [:read, :write, :binary]) do
      {:ok, %{pipe_name: pipe_name, pipe: pipe, task: nil, caller: nil}}
    else
      {:error, _} = error -> {:stop, error}
      {_, _num} -> {:stop, {:error, "mkfifo"}}
    end
  end

  def terminate(_, state) do
    Logger.warn("PipeWorker #{state.pipe_name} exit")
    state.task && Task.shutdown(state.task, :brutal_kill)
    Logger.warn("Pipe Task shut down")
    IO.inspect(state.pipe, label: "pipe")
    # :file.close(state.pipe) # blocks indefinitely no matter what. Shell becomes unresponsive.
    # Logger.warn("Pipe closed")
    File.rm!(state.pipe_name)
    Logger.warn("Pipe removed")
  end

  def handle_call({cmd, args}, {pid, _} = _from, state) do
    IO.inspect([state.pipe | args], label: "Pipe task args")
    task = Task.async(:file, cmd, [state.pipe | args])
    IO.inspect(task, label: "Pipe task")
    {:reply, task.ref, %{state | task: task, caller: pid}}
  end

  # This is never called?
  def handle_info({ref, result}, %{task: %{ref: ref}, caller: pid} = state) do
    IO.inspect({ref, result}, label: "Task result")
    send(pid, {__MODULE__, ref, result})
    {:noreply, %{state | task: nil, caller: nil}}
  end
end

Marked As Solved

josevalim

josevalim

Creator of Elixir

So this worked:

~/OSS/elixir[jv-basic-releases %]$ mkfifo test.pipe
~/OSS/elixir[jv-basic-releases %]$ erl
Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Eshell V10.0  (abort with ^G)
1> Fifo = open_port("test.pipe", [eof]),
1> receive
1> Msg -> io:format("Got ~p", [Msg])
1> end.
Got {#Port<0.5>,{data,"foo\n"}}ok

After typing the receive, I ran echo "foo" > test.pipe in another terminal. Could it be something related to permissions as the error message says?

Also Liked

josevalim

josevalim

Creator of Elixir

The solution is to use ports, which will also give asynchrony. See previous discussion here: Elixir vs Unix named pipe

I think it was recently announced that erlang supports pipes, but I cant recall if in port or file. Or I may be completely misremebering it. Anyway, a port should do nowadays.

rvirding

rvirding

Creator of Erlang

Well the file:read operation blocks by design. Two solutions have already been mentioned: put the read in a separate process which sends you a message when done and which you can kill when you give up on it; use ports which are non-blocking.

rvirding

rvirding

Creator of Erlang

I have a stupid question. You are reading with:

{:ok, <<protocol_pattern_match>>} = :file.read(fifo, protocol_size)

Do you write enough bytes into the fifo so this read can complete? I expect yes but it is always best to ask. Start with the simple questions first. :smile:

ConnorRigby

ConnorRigby

Nerves Core Team

Right. I chose the port method and it seems to be working well!

ConnorRigby

ConnorRigby

Nerves Core Team

Okay so while investigating this, i discovered something.

This code will hang forever no matter what. It doesn’t matter how much data
is written to the pipe, the data will never be read by Erlang/Elixir.

{:ok, file} = :file.open(path_to_fifo, [:read, :write, :binary])
{:ok, "hello world"} = :file.read(file, 11)
# in a different tab to `File.write(path_to_fifo, "hello world")`

This code however works fine. I suspect this is because the :raw option is a NIF now. (just a guess?)

# notice the `:raw` option.
{:ok, file} = :file.open(path_to_fifo, [:read, :write, :binary, :raw]) 
{:ok, "hello world"} = :file.read(file, 11)
# in a different tab to `File.write(path_to_fifo, "hello world")`

The only problem with opening in :raw mode is that i can’t use Task to call read/2 because the Task is not an owning process. I get the following error:

** (ErlangError) Erlang error: :not_on_controlling_process
        :prim_file.get_fd_data/1
        :prim_file.read/2
        (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
        (elixir) lib/task/supervised.ex:38: Task.Supervised.reply/5
        (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

I guess this isn’t really an Elixir issue, but an Erlang one but any help would be appreciated.

Where Next?

Popular in Questions Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
electic
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
idi527
I’ve been re-reading swift book again and noticed that multiline strings there don’t have a trailing line break, unlike in elixir iex(2)...
New

Other popular topics Top

Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
beno
I will often find my self writing things similar to: case some_value do nil -&gt; something() "" -&gt; something() _ -&gt; someth...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
Nvim
Elixir appears to be a superior language to Python. I don’t see any advantage of Python over Elixir. Are there any?
New

We're in Beta

About us Mission Statement