What concrete changes would need to be made to achieve Echo/Siri-level responsiveness?
All of the Vaani demos I’ve seen include substantial latency, such that a product using Vaani would immediately be outed as a second-class citizen at best. Is it possible to achieve competitive latency with our server-side Kaldi-based solution? What would it take to get us there, in actionable terms?
Good point, fwiw, the Mac OSX Chess app also has up to 6 seconds latency both in offline and in online mode, which is a lot.
Do you have numbers on Echo and Siri speed, though? I never used either of them but just came across this forum thread which reports significant latencies for Siri, at least for some users: http://www.ign.com/boards/threads/is-there-a-big-delay-from-when-you-ask-siri-something-to-when-u-get-an-answer.452701645/
We haven’t optimized the pipeline at all. There are a lot of parameters we have to tune.
The first step is to get all the relevant timestamps for the roundtrip and then we can start fixing the
bottlenecks.
I assume we will find some low hanging fruits.
-Gregor
I’ve heard that Cortana is faster than google now/siri/alexa, hitting < 1s response time consistently.