I Built a Jarvis-Style AI Dashboard

A screen on the wall that boots up like Iron Man's assistant, reads me a briefing out loud, and answers when I talk to it. It sounds like a toy. It turned out to be one of the most useful things I run, because under the hood it is not a gimmick. It is an AI agent with a face.

Why build this at all

Most mornings used to start the same way: open the calendar, open email, open the project board, and spend the first twenty minutes stitching together what actually matters today. That is twenty minutes of low-value digging before any real work starts.

The dashboard does that digging for me. I walk in, it greets me, and it says the three or four things I need to know in plain spoken sentences. Then it gets out of the way. That is the whole point: it is a multiplier on attention. It does the gathering so I start the day already briefed instead of assembling the brief myself.

The Iron Man look is mostly for fun. The value is the voice and the fact that it knows my real day.

How it actually works

There are three layers, and only one of them is interesting.

The screen is just a web page. A small program on the machine serves that page and also acts as a bridge: when the page wants an answer, the bridge passes the question to an AI agent and passes the reply back. The agent is the brain. It runs on a small server I control, it has my context, and it can actually look things up. That agent is an OpenClaw agent, the same open-source runtime we use to put assistants on WhatsApp and Telegram. Here it just happens to have a screen and a voice instead of a chat thread.

So the dashboard is not the product. It is a face on an agent you own. Swap the face for a phone, a speaker, or a chat app and the brain is the same.

The lessons that actually mattered

1. Write for the ear, not the screen

The first version read like a chatbot: bullet points, headers, the occasional stray ID. Spoken aloud, it was unbearable. The fix was to tell the agent that every reply is read out loud, so answer in three to five short, natural sentences, no lists, no markdown, no file paths, no IDs. The moment the persona was written for the ear, it started sounding like a person instead of a printout.

2. Pre-cache the briefing so it is instant

Asking the agent for a full briefing live takes a few seconds while it looks things up. A few seconds of silence feels broken. So a small job runs on a schedule, builds the briefing ahead of time, and saves it. When I walk up, the dashboard reads the cached version instantly, and only fetches live if the cache is empty. Instant beats fresh for something you hear every morning.

3. Clean the reply before it speaks

Agents sometimes leak their plumbing into a reply: a stray bit of tool output, a link, a long ID, a half-finished error. Fine on a screen, jarring out loud. A small cleanup step strips that noise before the text is spoken, while the chat panel keeps the full text. Speak the clean version, show the complete one.

4. Keep the keys off the screen

The browser page never holds the secret that talks to the agent. The local bridge injects it on the server side and only listens on the machine itself, reaching the agent over a private tunnel. A dashboard is the worst place to hardcode a token, because anyone who opens the page can read it. Tokens live in the bridge, never in the HTML. Learn from the version I had to clean up: a key in the page is a key in everyone's hands.

5. Make the voice feel alive

A few small touches did most of the work. It speaks sentence by sentence and starts generating the next line while the current one plays, so there is no long pause before it talks. It only acts when you say its name, so it ignores the room. It ignores its own voice so it does not answer itself. And it sends what you said about a second after you stop talking, so you are not racing a timer. None of these are fancy. Together they are the difference between a demo and something you actually use.

6. Ask the agent for JSON when you need structure

For the spoken briefing, plain sentences are perfect. For the on-screen agenda, I needed structured data. So the agenda request asks the agent to reply with only a JSON array of events, nothing else, and the dashboard parses that. Same agent, two formats: prose for the ear, JSON for the screen.

7. Make it boring and resilient

It starts on its own when the machine logs in, the tunnel reconnects itself if it drops, and the screen reopens after the machine is unlocked. A wall display nobody has to babysit is the goal. The least glamorous part, the auto-restart plumbing, is what makes it something you trust instead of something you keep restarting.

What I would skip

Voice kiosks are finicky. Microphones get blocked, browsers handle speech differently, and a screen that talks to a quiet office can feel strange. This is worth building only if you will genuinely use it every day. If you would glance at it twice and forget it, a plain chat agent on your phone is the better tool and a tenth of the effort.

And the honest one: keep your secrets server-side from day one. The cleanup that cost me the most time was pulling a key back out of a page where it never belonged.

The bottom line

The fun part is that it looks like Jarvis. The useful part is that it is an agent you own wearing a screen. The briefing, the voice, the answers, all of it rides on the same OpenClaw agent that could just as easily live in your pocket. The dashboard is one face. The brain is the thing worth building.

If you want the brain without the weekend of plumbing, that is exactly what we do.