Back to System Design
System Design
easy
mid

What are the UX challenges unique to AI interfaces, such as latency, hallucinations, and streaming?

AI UIs must handle high/variable latency (stream + show progress), nondeterminism (no fixed 'correct' UI), hallucinations (don't present output as authoritative — enable verification, citations, editing), trust calibration, error/refusal handling, cost awareness, and giving users control (stop, regenerate, edit).

7 min read·~10 min to think through

AI interfaces break assumptions normal UIs rely on — responses aren't instant, aren't deterministic, and aren't always correct. The UX has to be designed around that.

1. Latency — high and variable

AI responses take seconds, and the time varies per request.

  • Stream the output — render tokens as they arrive so the user sees progress immediately instead of a multi-second spinner.
  • Progressive feedback — typing indicators, "thinking…" states, skeleton-like affordances.
  • Set expectations — the UI should make slowness feel intentional, not broken.
  • Let users keep working — don't lock the whole UI during a generation.

2. Nondeterminism — no single "correct" output

Same input → different output each time.

  • You can't design a UI around a fixed expected response shape; it must handle variable length, format, and content.
  • Makes testing hard — test behaviors and guardrails, not exact strings.
  • Regenerate affordance — since outputs vary, let users get a different one.

3. Hallucinations — output can be confidently wrong

The hardest one. The model can produce plausible, fluent, false content.

  • Don't present output as authoritative. Frame it as AI-generated, not fact.
  • Enable verification — cite sources, link to references, show the underlying data when possible.
  • Keep humans in the loop — let users edit, correct, accept/reject, especially before consequential actions.
  • Calibrate trust — the UI should convey appropriate uncertainty, not false confidence.
  • Be especially careful in high-stakes domains (medical, legal, financial).

4. Streaming complications

Streaming helps latency but adds its own UX problems:

  • Partial content — must render incomplete markdown/code gracefully mid-stream.
  • Mid-stream errors — the stream can drop; keep partial output, offer continue/regenerate.
  • Layout shift — content growing as it streams; reserve/manage space.
  • Stop control — users must be able to halt a generation in progress.

5. User control & agency

  • Stop / cancel an in-progress response.
  • Regenerate, edit the prompt and retry, edit the output.
  • Conversation management — history, context, clearing.
  • Make it feel like a tool the user steers, not a black box.

6. Errors, refusals, safety

  • Content-filter refusals and safety blocks need graceful, non-alarming messaging.
  • Errors (rate limits, timeouts) communicated clearly with retry.
  • Set boundaries on what the AI will/won't do so users aren't confused.

7. Cost & efficiency awareness

  • AI calls cost money — the UX shouldn't encourage wasteful regeneration; debounce, confirm expensive actions.
  • Show usage/limits where relevant.

How to answer

"AI UIs violate the usual assumptions — responses aren't instant, deterministic, or reliably correct. Latency: stream tokens and give progressive feedback. Nondeterminism: design for variable output and offer regenerate. Hallucinations — the critical one: never present output as authoritative, enable verification with citations, keep humans in the loop to edit/approve, and calibrate trust honestly. Streaming adds partial-content and mid-stream-error handling. And users need control — stop, regenerate, edit — plus graceful refusal/error handling and cost-aware design. The throughline: design for uncertainty and keep the human in control."

Follow-up questions

  • Why is hallucination the hardest UX challenge, and how do you design around it?
  • How does streaming help latency but create new UX problems?
  • Why does nondeterminism make AI interfaces hard to test?
  • What controls should users have over an AI generation?

Common mistakes

  • Presenting AI output as authoritative fact with no verification path.
  • A multi-second spinner instead of streaming with progressive feedback.
  • No stop/regenerate/edit controls — a black box.
  • Not handling mid-stream errors or partial content gracefully.
  • Designing the UI around a fixed expected response shape.

Performance considerations

  • Streaming is the key perceived-performance lever — first token in ~1s beats a 10s wait. But it adds partial-render and layout-shift handling. Caching repeated prompts cuts both latency and cost.

Edge cases

  • High-stakes domains where a hallucination causes real harm.
  • Stream dropping mid-response.
  • Content-filter refusals.
  • Very long generations and cost concerns.
  • Conflicting outputs across regenerations.

Real-world examples

  • ChatGPT/Claude UIs: streamed responses, stop button, regenerate, edit, citations in retrieval-augmented modes.

Senior engineer discussion

Seniors enumerate the challenges — latency, nondeterminism, hallucinations, streaming complications, control, errors/refusals, cost — and single out hallucination handling as the hardest: never authoritative, enable verification, human-in-the-loop, honest trust calibration. The throughline they articulate is designing for uncertainty while keeping the user in control.

Related questions