How I Built a Voice-Controlled Local AI Agent from Scratch

A Streamlit UI shows a locally running voice-controlled AI agent.

It accepts microphone input or uploaded .wav, .mp3, and .m4a files.

Audio is transcribed with Groq's Whisper API using whisper-large-v3.

An LLM turns the text into JSON intent data; Ollama with llama3.2 is primary, with Groq LLaMA-3.3-70b as fallback.

The tools include write_code, create_file, summarize, general_chat, and compound, with file writes limited to output/.