Hi, I am e-Milka. And I am e-Milek. We explain it like a human, even though we aren't one.

e-Milki are two Polish voice companions we are building at the Dobre Państwo Foundation for adults with intellectual disabilities. Not for paperwork: for an ordinary day. Getting up, what to wear, is the cooker off, is the door locked, how to get there, a little shopping, the alarm. The project has just been accepted into the ElevenLabs Impact Program: dobrepanstwo.org/e-milki.

This post is about how it is built and what we have learned so far. Everything below runs on test data only; the pilot with real users starts in Q2 2027.

The conversation loop: ElevenLabs Agents with Claude as the brain

The assistant is an agent on the ElevenLabs Agents platform:

  • Listening: the platform's speech recognition (Scribe).
  • Understanding: the agent's LLM is Claude Sonnet 5.5. We started with a smaller, faster model, but it struggled with understanding and with Polish grammar for clock times (Polish inflects hour numerals, and getting „o siedemnastej" wrong is confusing for exactly the people we serve). Sonnet was clearly better in language and in safety, at the cost of 2–3 s of extra latency and roughly 25% more per minute.
  • Speaking: a calm Polish voice, TTS eleven_flash_v2_5 at speed 0.85. One measured surprise: the newer default turbo model ignored the speed setting in our tests (1.0 and 0.8 produced the same 4.91 s clip), so for slow, clear speech we stayed on Flash v2.5.

Two small settings matter a lot for this audience:

  • Patient turn-taking: 15 s turn timeout, hang up only after 60 s of silence. People need time to answer.
  • A soft timeout filler: after 2.5 s the agent says „Chwileczkę." („Just a moment.") so the LLM latency does not feel like the call dropped.

The prompt: fewer words, more checking

The hard rules that survived our test conversations:

  • at most three short sentences per answer, one step at a time;
  • check understanding before moving on, and repeat the same thing more simply when asked to go slower;
  • current time from {{system__time_utc}} plus the Polish offset, spoken with the correct inflection;
  • health and money: no advice, always point to a trusted person.

We test with text simulations of conversations first (the platform can simulate a user), and only then with voice. It saves agent minutes and makes prompt regressions easy to spot.

The web page: three.js characters drawn from dots

The page lets e-Milka and e-Milek introduce themselves, alternating two voices. A few decisions that kept it light and accessible:

  • One source of content. A single TypeScript module holds the scene texts. The same file feeds the page, a Markdown version for crawlers and AI assistants, and the narration script, so they never drift apart.
  • Lazy 3D and audio. The page itself is about 17 kB. The three.js scene (about 455 kB) and the sound load only after the visitor clicks „Listen with voice".
  • Characters made of dots. Each speaker is a simple dot drawing; the mouth of whoever is talking moves in sync with the narration using known timings for each line.
  • Narration generated once. We render the narration with ElevenLabs voices and check every file with a speech-to-text pass, to make sure names like „e-Milka" come out right. Audio files carry a version parameter in the URL, because they are cached for a week.
  • Quiet background. A soft Web Audio chord plays only under the voice and fades to silence after each line.

What we want to learn in the pilot

Two questions we cannot answer from a desk: will speech recognition understand the speech of people with intellectual disabilities, and which voice and speaking rate are truly understandable for them? That is what the pilot is for, and what the ElevenLabs Impact Program support helps us test properly.

Listen to e-Milki here: dobrepanstwo.org/e-milki

Written with AI assistance and edited by the Dobre Państwo Foundation. The e-Milki voices are synthetic (ElevenLabs).