RealChar
by Shaunwei
An open-source platform for building and talking to customizable AI characters in real time, over voice or text, across a web app, iOS app, and terminal client.
Run with Docker Compose
cp .env.example .env && docker compose up
About
RealChar is a full-stack project for creating a character — a persona, backstory, and voice — and then holding a live conversation with it rather than a generic chat window. The pipeline runs Whisper for speech-to-text, routes the transcript through a configurable LLM backend (OpenAI GPT-3.5/4, Anthropic Claude, or Llama 2 via Anyscale), stores conversation memory in a Chroma vector database, and speaks the reply back with ElevenLabs, Google TTS, or Edge TTS — with ElevenLabs also handling voice cloning so a character can have a distinctive voice rather than a generic TTS default.
The same backend serves three different front ends from one codebase: a React web client over WebSockets for low-latency voice conversation in the browser, a native Swift iOS app, and a terminal client for a lighter, keyboard-only interaction. Every component — the LLM, the STT/TTS engine, the vector store — is swappable, and the whole stack ships as a Docker Compose setup, so self-hosting doesn't require assembling the pieces by hand.
Key features
- Real-time voice conversation pipeline: Whisper STT, configurable LLM, TTS reply
- Voice cloning via ElevenLabs for distinctive per-character voices
- Three front ends from one backend — web, native iOS, and terminal
- Swappable LLM backend (OpenAI, Anthropic Claude, or Llama 2 via Anyscale)
- Chroma vector database for persistent character memory
- Docker Compose deployment for self-hosting the full stack
Use cases
- Building a branded AI companion or mascot with a persistent voice and personality
- Prototyping a voice-first assistant without wiring together STT, LLM, and TTS services separately
- Running a local, self-hosted character chat instead of a third-party companion app
- Comparing how different LLM backends handle the same character persona and memory
Available tools
Character builder
Defines a character's persona, backstory, and voice configuration.
Realtime voice pipeline
Chains Whisper STT, the configured LLM, and TTS into a live spoken conversation loop.
Chroma memory store
Persists conversation history as embeddings so a character can recall prior context.
Frequently asked questions
Does RealChar expose an MCP server?
No — it's a self-contained conversational AI platform with its own web, iOS, and terminal clients, not an MCP tool-call server for other agents to call into.
Can I swap out the LLM or voice engine?
Yes — the backend is modular, supporting OpenAI, Anthropic Claude, or Llama 2 for the model, and ElevenLabs, Google TTS, or Edge TTS for speech output.