pavel-sev

pavel-sev

IO problems - (IO.StreamError) error during streaming: unknown POSIX error

Hello

I am trying to deal with stdio and face strange (for me) behavior when reading bytes. To reproduce it I did simple one line bash script:

elixir -e ':binary.list_to_bin( for n <- 0..255, into: [], do: n) |> IO.binwrite()' | elixir -e 'IO.binstream(:stdio, 1) |> Enum.each(&(&1))'

and the error:

** (IO.StreamError) error during streaming: unknown POSIX error
    (elixir 1.14.5) lib/io.ex:752: IO.each_binstream/2
    (elixir 1.14.5) lib/stream.ex:1625: Stream.do_resource/5
    (elixir 1.14.5) lib/enum.ex:4307: Enum.each/2
    nofile:1: (file)
    (stdlib 4.3) erl_eval.erl:748: :erl_eval.do_apply/7

Actually it works without Enum.each but I need this data )
Did I missed some thing or maybe there is better way to read from stdio?

Elixir 1.14.5-otp-25
Erlang 25.3
OS Ubuntu 22.4

#io

Marked As Solved

al2o3cr

al2o3cr

There are two problems with this code:

  • the code that writes doesn’t actually produce the bytes you expect

  • the code that reads is trying to read binary data from a device set to UTF8 encoding


The first issue, as the documentation for IO.binwrite mentions:

Important: do not use this function on IO devices in Unicode mode as it will write the wrong data. In particular, the standard IO device is set to Unicode by default, so writing to stdio with this function will likely result in the wrong data being sent down the wire.

The actual data that the first half of that pipeline prints out is 384 bytes long, because each byte of the second half of the data is translated to the corresponding Unicode character (so 128<<194, 128>>

Calling :io.setopts(:standard_io, [encoding: :latin1]) before binwrite avoids this behavior and writes exactly the 256 bytes expected.


The second issue is that IO.binread also behaves unexpectedly with a Unicode device. Reading a single byte from it will fail if bytes farther along in the stream are invalid Unicode. so:

echo $'\x01\x02' | elixir -e ':file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:ok, <<1>>}
{:ok, <<2>>}


# but this doesn't work even though the first two bytes are both OK
echo $'\x01\x02\xFF' | elixir -e ':file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:error, :collect_chars}
:eof

This {:error, :collect_chars} is swallowed by higher levels of the IO machinery and produces the “unknown POSIX error” explanation you’re seeing.

As in the first case, setting the encoding correctly produces the correct result:

echo $'\x01\x02\xFF' | elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); :file.read(:standard_io, 1) |> IO.inspect; :file.read(:standard_io, 1) |> IO.inspect'

# prints
{:ok, <<1>>}
{:ok, <<2>>}

(note: this is the same collect_chars error shape as in this ticket that @josevalim just filed, but I don’t believe it’s directly related)


Putting both together does what you want:

elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); :binary.list_to_bin( for n <- 0..255, into: [], do: n) |> IO.binwrite()' | elixir -e ':io.setopts(:standard_io, [encoding: :latin1]); IO.binstream(:stdio, 1) |> Enum.each(&IO.inspect/1)'

# prints
<<0>>
...
<<255>>

Also Liked

al2o3cr

al2o3cr

What specifically do you mean by “it”?

What specifically do you mean by “did not work”?

You can see the code you’re trying and the error message you’re getting, but the rest of us can’t.

Also, consider opening a new thread if you have a new problem.

Where Next?

Popular in Questions Top

JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
pgiesin
This should be a simple problem but I just can’t seem to figure it out. I have a standalone Elixir app that won’t find the database. Dep...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
chrisalley
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New

Other popular topics Top

Qqwy
Update: How to use the Blogs &amp; Podcasts section You can post links to your blog posts or podcasts either in one of the Official Blog...
3268 119930 1237
New
JDanielMartinez
Hi! May someone helps me, please! I have two apps into an umbrella project: the first one is Database, which manages queries, and the se...
New
yawaramin
In the Dialyzer docs ( http://erlang.org/doc/man/dialyzer.html#requesting-or-suppressing-warnings-in-source-files ), there is a way to tu...
New
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 27727 240
New
9mm
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
New
lucidguppy
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement