niku

How to manipurate bitstring(not binary) such as making a value, concatinating, converting into integer?

As you know, Erlang and Elixir have a good specifications/functions for binary. However, I haven’t found it for bitstring. Here is some examples I ran into. I’m happy to hear your thoughts and solutions. Please let me know

Making a value

Sometimes you desire making a value as you read such as “00010010_10101011”.
It’s easy for binary with using integer literal starting 0b and :binary.encode_unsigned/1.

iex(1)> a = 0b00010010_10101011
4779
iex(2)> b = :binary.encode_unsigned(a)
<<18, 171>>
iex(3)> bit_size(b)
16
iex(4)> i b
Term
  <<18, 171>>
Data type
  BitString
Byte size
  2
Description
  This is a binary: a collection of bytes. It's printed with the `<<>>`
  syntax (as opposed to double quotes) because it is not a UTF-8 encoded
  binary (the first invalid byte being `<<171>>`)
Reference modules
  :binary
Implemented protocols
  Collectable, IEx.Info, Inspect, List.Chars, String.Chars

However, :binary.encode_unsigned/1 always makes a binary, I haven’t found the best practice for making a bitstring. Here is some techniques I use currently:

iex(1)> <<0::1, 1::1, 0::1, 1::1, 1::1>>
<<11::size(5)>>
iex(2)> # It's a little bit verbose but It works well
nil
iex(3)> <<0b01011::5>>
<<11::size(5)>
iex(4)> # It's simple but requires size explicitly
nil

and also using sigil which I have implemented.

iex(7)> import BitsSigils
BitsSigils
iex(8)> ~b(01011)
<<11::size(5)>>
iex(9)> # I like this, but needs to implement some codes
nil

Concatinating

For binary, We use <>/2 to concatinate.

iex(1)> a = "abc"
"abc"
iex(2)> b = "123"
"123"
iex(3)> a <> b
"abc123"

For bitstring, I can’t use both <>/2 and just <<>>/2.

iex(1)> a = <<1::1>>
<<1::size(1)>>
iex(2)> b = <<0::1>>
<<0::size(1)>>
iex(3)> a <> b
** (ArgumentError) argument error
    (stdlib) eval_bits.erl:101: :eval_bits.eval_exp_field1/6
    (stdlib) eval_bits.erl:92: :eval_bits.eval_field/3
    (stdlib) eval_bits.erl:68: :eval_bits.expr_grp/4
    (stdlib) erl_eval.erl:484: :erl_eval.expr/5
    (iex) lib/iex/evaluator.ex:257: IEx.Evaluator.handle_eval/5
    (iex) lib/iex/evaluator.ex:237: IEx.Evaluator.do_eval/3
iex(3)> <<a, b>>
** (ArgumentError) argument error
    (stdlib) eval_bits.erl:101: :eval_bits.eval_exp_field1/6
    (stdlib) eval_bits.erl:92: :eval_bits.eval_field/3
    (stdlib) eval_bits.erl:68: :eval_bits.expr_grp/4
    (stdlib) erl_eval.erl:484: :erl_eval.expr/5
    (iex) lib/iex/evaluator.ex:257: IEx.Evaluator.handle_eval/5
    (iex) lib/iex/evaluator.ex:237: IEx.Evaluator.do_eval/

So, I use <<>>/2 with segment types( ::bitstring).

iex(3)> <<a::bitstring, b::bitstring>>
<<2::size(2)>>

Converting into integer

For binary, we have nice functions thanks for :binary module. Unfortunary, It doesn’t work for bitstring.

iex(1)> :binary.decode_unsigned(<<1, 255>>)
511
iex(2)> :binary.decode_unsigned(<<1::1, 255>>)
** (ArgumentError) argument error
    (stdlib) :binary.decode_unsigned(<<255, 1::size(1)>>)

I use pattern matching converting bitstring to integer. Please note you can only use integer or variable for size of bitstring in <<>>/2 .

iex(1)> a = <<1::1, 255>>
<<255, 1::size(1)>>
iex(2)> <<x::integer-size(bit_size(a))>> = a
** (CompileError) iex:2: size in bitstring expects an integer or a variable as argument, got: :erlang.bit_size(a)
    (elixir) src/elixir_bitstring.erl:197: :elixir_bitstring.expand_each_spec/5
    (elixir) src/elixir_bitstring.erl:168: :elixir_bitstring.expand_specs/6
    (elixir) src/elixir_bitstring.erl:41: :elixir_bitstring.expand/8
    (elixir) src/elixir_bitstring.erl:10: :elixir_bitstring.expand/4
iex(2)> b = bit_size(a)
9
iex(3)> <<x::integer-size(b)>> = a
<<255, 1::size(1)>>
iex(4)> x
511

2 comments

#binary #bitstring

14 8620 2

2019-05-25 17:27:05 UTC

Most Liked

dimitarvp

Based on a quick search in hex.pm I found these might be of help for you:

bitmap (Erlang)
bit_field_set (made by @gausby)
ex_bin (made by @zcking)
codec (a bit higher level: hierarchical binary protocols)
bitset
bite

Didn’t notice any of them offering a merge of two bitstrings but maybe it can be made to work with their functions.

Post #3

jmitchell

Nice overview of different challenges and approaches. Recently I’ve been asking some of these questions too.

Is the bitstring you expect the same as the :binary.encode_unsigned/1 result, except with the leading 0 bits removed? If so, when you’d prefer to start from an integer rather than explicit bits using your nifty BitSigils module, this may help:

def trim_leading_zeros(<<0::1, bs::bitstring>>) do
  trim_leading_zeros(bs)
end
def trim_leading_zeros(bs) when is_bitstring(bs), do: bs

Usage:

iex(5)> 4779 |> inspect(base: :binary)
"0b1001010101011"
iex(6)> 4779 |> :binary.encode_unsigned() |> trim_leading_zeros() |> inspect(base: :binary)
"<<0b10010101, 0b1011::size(5)>>"

This is the approach I’ve landed on too. You may appreciate this flatten_bitstrings/1 function. It’s similar to IO.iodata_to_binary/1, except it works on bitstrings as well as binaries.

def flatten_bitstrings([]), do: <<>>
def flatten_bitstrings([x]) when is_bitstring(x), do: x
def flatten_bitstrings([x]) when is_list(x), do: flatten_bitstrings(x)
def flatten_bitstrings([x | [y | zs]]) do
  <<flatten_bitstrings([x])::bitstring,
    flatten_bitstrings([y])::bitstring,
    flatten_bitstrings(zs)::bitstring>>
end

Usage:

iex(12)> flatten_bitstrings([<<1::1>>, <<0::1>>])
<<2::size(2)>>
iex(13)> flatten_bitstrings([[[<<1::1>>], [[<<0::1>>]]]])
<<2::size(2)>>

Assuming you still only care about unsigned integers, I’d do the inverse of what I suggested for converting from bitstrings to integers. That is, pad the front of the bitstring with zeros until it is a binary, and then use :binary.decode_unsigned/1.

def pad_leading_zeros(bs) when is_binary(bs), do: bs
def pad_leading_zeros(bs) when is_bitstring(bs) do
  pad_length = 8 - rem(bit_size(bs), 8)
  <<0::size(pad_length), bs::bitstring>>
end

Usage:

iex(25)> encoded = 4779 |> :binary.encode_unsigned() |> trim_leading_zeros()
<<149, 11::size(5)>>
iex(26)> decoded = encoded |> pad_leading_zeros() |> :binary.decode_unsigned()
4779

Hope that helps