For those interested in the subject, I did some basic tests with https://github.com/mozilla/DeepSpeech

$ deepspeech --model deepspeech-0.5.1-models/output_graph.pbmm --alphabet deepspeech-0.5.1-models/alphabet.txt --lm deepspeech-0.5.1-models/lm.binary --trie deepspeech-0.5.1-models/trie --audio audio/8455-210777-0068.wav 
Loading model from file deepspeech-0.5.1-models/output_graph.pbmm
TensorFlow: v1.13.1-10-g3e0cc5374d
DeepSpeech: v0.5.1-0-g4b29b78
2019-09-27 01:53:25.693154: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: UnwrapDatasetVariant
2019-09-27 01:53:25.693187: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "CPU"') for unknown op: UnwrapDatasetVariant
2019-09-27 01:53:25.693200: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: WrapDatasetVariant
2019-09-27 01:53:25.693211: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "CPU"') for unknown op: WrapDatasetVariant
Loaded model in 0.0174s.
Loading language model from files deepspeech-0.5.1-models/lm.binary deepspeech-0.5.1-models/trie
Loaded language model in 1.38s.
Running inference.
your power is sufficient i said
Inference took 1.562s for 2.590s audio file.

It looks promising and is even faster than expected. And with a compatible card, it is possible to run on GPU

Thanks again @idi527 for the link. Now I need to glue ffmpeg and deepspeech with Elixir to have the beginning of a working solution.

Hello Christian, I’m curious about your findings as well. My Gist so far:

• Whisper is great at speech recognition, has timing issues though and I’m not yet sure how to deal with silences in speech (I want to hide subs for example when nobody is talking… use VAD Voice Activity Detection on top? Dunno ) I ran my own and also on replicate.com, which is pretty fast (1h file in 90 secs)
• Sonix feels like the goat to me. Similarly good in regards of speech as whisper, but way better timings, speaker diarization, etc, etc. BUT: expensive!
• rev.ai - is what I’m currently using for my Video CMS. It’s actually pretty good and timings are very accurate. It’s much cheaper than Sonix and I feel gives me the best bang for the buck right now (I’m currently torn between sonix and rev )
• aws transcribe - feels “ok”. timings are good, speech in my examples maybe a little less accurate than rev.ai
• fireflies - Cool tool, didn’t play with it that much yet. I think they might use whisper under the hood, as I see a bunch of weird words that whisper was able to catch successfully (none of the others did), and they caught them too. They also have timing issues.
• deepgram - feels very similar what aws and rev give me on my first few tries. pretty cheap though and lots of features, so I will play some more with it. I give them props for good DX as well.

Anyway, that’s quickly from me. Would love a DM from you to see what your experiences are. Thank you!

I am using erlport to communicate with python scripts.
I added poolboy to be able to process multiple files concurently, but limited to 10 workers.

Here is an example for the sample file, in french…

iex(9)> WhisperStt.speech_to_text path, "fr"
{:ok,
 %{
   language: "fr",
   segments: [
     %{end: 1.0, start: 0.0, text: "C'est le machine-machin."},
     %{
       end: 3.0,
       start: 1.0,
       text: "La machine-machin est la plus mignonne de l'automobile."
     },
     %{end: 4.0, start: 3.0, text: "Il a des détails très tristiques."},
     %{
       end: 5.0,
       start: 4.0,
       text: "Le tristique, la position, le pain, le poids,"
     },
     %{end: 6.0, start: 5.0, text: "plus un mètre incroyable."},
     %{end: 7.0, start: 6.0, text: "Le mètre machine, le poids, le place,"},
...

I have python installed on the server, and I have a requirements.txt for the plugin.

@maz OpenAI’s Whisper works great and it is open source (python):

FYI: This just popped up on the changelog.com weekly newsletter this morning.

I don’t know anything more about it, but thought it may be of interest.