“Side unclear”: turning a field observation into conversation design.

400+ patients, one recurring gesture

Over about a year I tested our AI physician assistant, on the mobile app and the web portal, with 400+ patients. Settings included Swecha’s free monthly medical camp in Hyderabad, where around 600 patients come in on the first Sunday of each month and a Pranik team is deployed, plus clinics, low-income settlements and apartment communities across Hyderabad, with some in Andhra Pradesh and Mumbai. Young and old users alike.

I sat next to people and watched them use the product. That is where the decisions came from.

Two moments stayed with me. An elderly woman from a low-income family cried while explaining her problems to the AI avatar. It showed how much trust people place in it, and it pushed us to build empathy into how the AI speaks. The other was smaller and kept repeating: patients pointed at their body to show where it hurt, on calls with no camera.

Why it was hard

On an audio call, “it hurts here” carries no information about side. But a language model wants to produce a complete answer, so it filled the gap and assumed a side.

Left-right inversion is one of the patient-safety categories in our evals rubric. A case sheet that says left knee when it is the right knee is not a typo. It is a clinical risk, and the failure is silent: nothing in the output tells the doctor it was a guess.

Options and trade-offs

Two ways to fix a model that can’t see

Option A

Give the AI eyes: a camera layer

• +The model could see where patients pointed
• −Left-right flipped depending on the device
• −The fix needed per-device calibration
• −More latency, shallower reasoning

Killed on evidence

Option B

Make the AI better at asking

• +Works on any phone, audio only
• +Honest about what it doesn’t know
• −Some sides it used to guess right are now flagged
• −A few more turns in the conversation

Shipped

What we shipped

The change was a prompt and guardrail change, designed as a conversation. The failure UX matters as much as the happy path: when the AI can’t get a clear answer, it hands the uncertainty to the doctor instead of hiding it.

Guardrail · say what you can’t see, then ask

Prompt and guardrail change, designed as a conversation

1. Patient points at the pain. “It hurts here.” The call is audio only.
2. The AI says what it can’t do. “I can’t see you on this call.”
3. It asks for the exact site, in words. Left or right, and where.
4. Named clearly? Record the exact site on the case sheet.
5. Still unclear after 2–3 tries? Flag “side unclear” to the doctor. Never guess.

Same call, two behaviors

Illustrative dialogue, not a real transcript

BeforeThe model fills the gap with a guess

1. Patient It hurts here, doctor. Right here.
2. AI Noted: pain in the left shoulder.
3. Record Case sheet → Left shoulder pain

AfterThe model says it can’t see, then asks

1. Patient It hurts here, doctor. Right here.
2. AI I can’t see you on this call. Is the pain on your left side or your right side?
3. Patient This side… the side I hold my phone.
4. AI Thank you. Is that your left hand or your right hand?
5. Patient I’m not sure, it’s here.
6. Record Case sheet → Shoulder pain · SIDE UNCLEAR, confirm in person

We tested it extensively at medical camps, where it comes up most with elderly patients and orthopedic cases.

Result, and the trade-off

Wrong-side errors are now under 10% in camp testing. The trade-off was deliberate: some sides the model used to guess correctly are now flagged instead. In clinical work, an honest “unclear” beats a confident wrong answer.

What it taught me

A field observation turned into a product decision: an observation, a tempting technical fix, evidence that killed it, and a smaller, humbler design that worked. It echoes something I studied at a bus depot during my master’s: designing information for anxious people who don’t behave the way the system expects. That study sits in the mission archives.