lawik

lawik

Nerves Core Team

Blog Post: Voice Activity Detection in Elixir with Membrane

Building on other people’s work I bashed things together and suddenly I can know when someone is speaking using Elixir and Membrane.

Most Liked

nefty

nefty

Great post! In case anyone else runs into the same issue, the latest silero_vad.onnx on their GitHub repo (version 5) expects different inputs than the version used in this post (version 4). The inputs h and c are now combined into a single state tensor: {"state", "Float32", [2, nil, 128]}
Here is the updated code:

init_state = %{state: Nx.broadcast(0.0, {2, 1, 128})}

live_audio
|> Kino.Control.stream()
|> Kino.listen(init_state, fn
  %{event: :audio_chunk, chunk: data}, %{state: state} ->
    input = Nx.tensor([data])
    sr = Nx.tensor(16_000, type: :s64)
    {output, state_n} = Ortex.run(model, {input, state, sr})
    prob = output |> Nx.squeeze() |> Nx.to_number()
    row = %{x: :os.system_time(), y: prob}
    Kino.VegaLite.push(chart, row, window: 1000)

    {:cont, %{state: state_n}}
end)
lawik

lawik

Nerves Core Team

Thanks! Linked to your post in a small update and also linked the original model that works with the code I had.

delitrem

delitrem

Awesome, very nice work!

By the way, I just remembered about 14 years ago I had a startup, where one of components was a gateway from real CB amateur radio (27Mhz) to our service. It was of course built with Erlang and used some magic around home-grown sox driver, which also had voice/sound activity detection.

P.s.: if I owned the time-machine that times, I would use your solution. :slight_smile:

Where Next?

Popular in Blog Posts Top

brainlid
You are storing some Phoenix LiveView state in the browser. You want to retrieve that saved state as early as possible to improve the use...
New
ErlangSolutions
An infographic that compares Erlang, Elixir, and Go's strengths in the respect to the programming languages' concurrency, reliability, sc...
New
AstonJ
I think <span class="hashtag-icon-placeholder"></span>liveview is going to be a big topic in the foreseeable future, so wondering whether...
New
marcelo
Long story short, over the past years we’ve been scaling our operation quite a lot at Unbabel. Most of our codebase is Python and some of...
New
paulanthonywilson
https://furlough.merecomplexities.com/elixir/otp/tdd/2021/03/18/test-driving-otp-creating-a-registry-with-expiring-entries.html Followin...
New
brainlid
I love LiveView. Navigating between views is so fast! This quick tip makes navigating feel instantaneous by adding a split-second delay b...
New
jordiee
https://medium.com/@jpiepkow/team-based-login-with-accesspass-be236d4bd7dd
New
New
New
marcin
Hi! :wave: I wanted to refresh my knowledge on how to mix phx.gen.auth with local password users db, as well as OAuth providers such as ...
New

Other popular topics Top

belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
977 41022 311
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
AstonJ
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New

We're in Beta

About us Mission Statement