Hacker Newsnew | past | comments | ask | show | jobs | submit | cjus's commentslogin

I built audio-chat-box to give a program a voice without sending audio anywhere. It reads text aloud with Kokoro, transcribes with NVIDIA's Parakeet TDT 0.6B v2, and can listen hands-free: your program streams microphone samples, Silero VAD finds each thing said, and Parakeet transcribes it when the speaker pauses. It all runs locally through sherpa-onnx.

To try it without building anything, there's a small browser demo in Docker. You speak, it shows what it heard, and it reads it back in any of Kokoro's 54 voices:

  docker run --rm --platform linux/amd64 -p 127.0.0.1:8080:8080 pnxstudios/audio-chat-box-demo
then open http://localhost:8080 (that address, not 127.0.0.1). The models are inside the image, so it needs no network, account or API key. It's amd64 only for now: on Apple Silicon it runs under emulation, about 4 s to speak a phrase, against under 1 s built natively.

It's a separate process, not a library. Your program starts it and exchanges JSON messages over stdin/stdout, one per line, so any language that can start a process can use it. It never touches the mic or speakers itself; your program records and plays. There's also a practical reason: sherpa-onnx's build includes eSpeak NG, which is GPL-3.0. As a separate program it's GPL, with the source of the native libraries it links published as a release, but your app starts it rather than linking it.

A few details:

- Hotwords: your program says which words to expect, and it favours them where the audio is ambiguous. It grew out of a chess app, so "knight" over "night".

- Speed: on an M1 Max, built natively, both models load in about a second, and a sentence is spoken in under one.

- Models: about 835 MB, downloaded once on first use and checked against SHA-256s built into the program. --offline never downloads.

- Protocol: fully documented in docs/protocol.md. The Docker demo's web server is itself a small client of it (demo-web/), if you want a worked example.

It began as the voice of Solrac (https://solrac-chess.com), an AI chess coach I'm building. Its listening was tuned on chess moves said at speed-chess pace. I took out everything chess-specific so any project can use it. I wrote it with Claude Code as a pair programmer; the commits say so.

Current limits: using it from your own program means building it with Rust (the Docker image is a demo), transcription is English only, and it handles one listen at a time. I'd love feedback, especially from anyone wiring voice into an agent or assistant.


Hey everyone. This may be a bit out there. I'm a developer and certified instructor in Qigong - an ancient mind body practice. Some months ago I decided to apply AI to create modules that support the exploration of these mind body practices. Would love some feedback.


Reading through the comments here I'm realizing that my single biggest error may have been the use of the word "Universal" :-D When you consider the spec as a "basis for agreement" between distributed application authors and not a unified theory of messaging - then the spec becomes a lot clearer.


The spec describes a message format and isn't intended to be a formal Internet RFC. As such, the thought was to leave definitions up to users rather than dictate things like priority ranges.


Thanks for this. The need for canonical JSON is perhaps the best reason for dropping the signature field from future versions of the spec. However, because only the `to` `from` and `body` fields are required there's no need avoid the format - just don't use the signature field unless the body field contains a single field with serialized data. Certainly, that use would need to be clearly documented.

Again your point is valid and will likely result in the depreciation of the signature field.

Thanks for taking the time to offer feedback.


+1 for mission statement, use case.

The point about inconsistent formatting is that by aligning a team(s) on how a message is formatted groups can avoid the introduction of multiple message formats between distributed services.

It's true the body field does introduce inconsistency and that's left for the application developers to resolve. The envelope fields are intended to be used in queuing and routing situations.

I disagree with "The destination doesn't need to be in the message". What about the use case where a message is forwarded or moves through proxy services?

+1 for metadata being larger than the payload :-D - Can't debate that. The only required fields are 'to', 'from' and 'body' and the short form of UMF can be used. Still, we've encountered situations where that metadata is still larger than the payload. But that doesn't completely invalidate the presence of the envelope in a distributed application.

Thanks, I really appreciate the time you took to offer feedback!


In an earlier version of the docs I actually had the spec in the readme but felt the sheer size was enough to send folks running. Using the readme to describe the spec (which I realize needs work!) allowed for a clean separation.


You should probably link to the spec or something in the README so it's easier for people to find.


I've always liked s-expressions, unfortunately, it hasn't caught on in the circles I travel.


Hahaha. My dude, it's a damn good thing I'm not applying for your startup! And a great thing that my current and past employers didn't think I should resign. So you know what? I'm going to keep up this charade for a bit longer. Besides, the pay is great! :-D

The conical issue with regards fields and signatures is definitely and without question a valid point and should probably be removed from the spec.

You raise good points. I'll definitely reconsider your feedback in future iterations!

Your point about a logging field is interesting, but the MID field could be used in an out of band error acknowledgment.

Protobuf is great - but what if you don't need or want it?

Thanks for taking the time to comment!


Ah Linus - enough said. The UMF spec wasn't intended to be a standards paper. You might be able to tell it isn't formatted as such. Based on the comments in this post - it has at the very least helped spark interesting feedback and debate.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: