Session abstract
What you’ll learn
Deploying an LLM feature is table stakes. Knowing whether it's actually working in production — that's the hard part most teams skip. This talk covers the observability layer your AI application needs but probably doesn't have yet: tracing individual inference calls end to end, tracking token costs and latency across model versions, detecting silent failures before users do, and correlating model behavior back to real user outcomes. We'll walk through the specific metrics that matter, the instrumentation patterns that actually scale, and the failure modes that only appear after you go live. Everything covered is tool-agnostic and grounded in real production experience, not demos built the night before. If you're shipping AI features and relying on "no error in the logs" as your signal that things are fine, this talk is for you.