Ship: AI-Native Product
The Level 5 finale - streaming UX, cost/latency budgets, AI observability, kill switches - shipped publicly.
Before this: AI Security and Evaluation
Objective
Combine every Level 5 skill into one public delivery: a grounded natural-language feature on your Ledger with streaming responses, enforced budgets, per-request AI observability, graceful degradation and a kill switch - evaluated pre-ship and smoke-tested post-deploy.
Vocabulary
The words to use when directing your agent.
- streaming response
- Delivering completion tokens progressively so users watch text arrive instead of staring at spinners.
- latency budget
- The maximum acceptable wait per interaction, enforced with timeouts and stream deadlines.
- cost budget
- Per-request and per-period ceilings on token spend, enforced in code.
- AI observability
- Per-request logs of prompts (sanitized), completions, token counts, latencies and computed costs.
- kill switch
- Environment-level instant disable of the AI path, falling back to non-AI behavior without redeploy.
Understand it
The streaming UX contract
Streams transform perceived performance: first tokens arriving within moments beat perfect answers arriving after dead air. Contract details matter: render partial text safely (incomplete markdown included), keep cancel buttons functional mid-stream, handle cut-off streams gracefully, and preserve accessibility - screen readers need considered aria-live treatment, not raw token soup. Perceived speed is a design achievement, not an accident.
Budgets, degradation and the launch checklist
Latency budget: deadline enforced server-side; streams that exceed it terminate with partial-plus-honest-note. Cost budget: per-request token ceilings and per-period aggregate caps; approaching limits triggers degradation (smaller contexts, cached answers) BEFORE hard rejection. Pre-launch: eval suites green, red-team pass current, kill switch tested LIVE, dashboards watching SLOs. Post-launch: smoke scripts exercising the AI path in production, rollback rehearsed - your Level 3 rhythms, now with GPUs somewhere behind them.
Core concepts
AI features fail differently - design those failures
Traditional features mostly succeed instantly or error clearly. AI features add slow-successes, confident-wrongs and capacity-dependent availability. Shipped AI products design FOR this: streams starting fast so waits feel shorter, timeouts bounding worst cases, degraded modes (cached answers, keyword fallback) keeping features useful, and honest labels when answers run on reduced quality.
Every generation is a bill and a log line
Production AI means per-request economics and forensics: token counts logged, costs computed from configured rates, latencies tracked, prompts sanitized before logging. Budgets ENFORCE in code - reject or degrade before exceeding ceilings, not after the invoice arrives. What you cannot see, you cannot cap.
Examples
Answers that stream, budgets that bite
'How much did I spend on dining last month?' starts rendering within moments, cites its source rows, finishes comfortably inside deadline. Behind scenes: token counts logged, cost computed, deadline armed, cache primed for repeat asks. Second identical question serves from cache - flagged as such - costing approximately nothing.
The flip that saved the evening
Provider incident degrades answer quality during peak hours. Kill switch flips the feature to keyword-search fallback: slightly dumber, fully honest labeling, zero downtime drama. Postmortem next morning uses per-request logs to quantify impact precisely. Ungated deployments would have spent the evening apologizing on social media instead.
Common mistakes
Common mistake
Shipping the AI path without budgets because 'usage will be small at first'.
Fix
Small usage grows or goes viral unexpectedly. Enforce ceilings from commit one - retrofitting budgets after an invoice shock is triage, not engineering.
Common mistake
Logging full prompts containing user data, or logging nothing at all.
Fix
Sanitize before persisting: strip personal fields, hash identifiers, retain diagnostic structure. Both extremes - forensic drought and privacy flood - are incidents waiting for calendars.
Working with your AI agent
Phase the engagement like your Level 2 capstone: brief and budgets agreed FIRST (latency targets, cost ceilings, degradation ladder), then build, then evaluate (eval + red-team suites green), then deploy rehearsal with kill-switch test, then production smoke. Demand completion evidence at every phase gate - this finale certifies the whole curriculum's working style, AI-flavored.
Build it
The module’s real project. Your agent builds; you direct and inspect.
Project: Ledger Copilot, Shipped
The curriculum's final boss: ship Ledger Copilot publicly - natural-language questions answered from YOUR expense data via your RAG stack, streaming UI, enforced latency/cost budgets, sanitized AI observability, kill switch, evaluated pre-ship and smoke-tested live with demo access for visitors.
Requirements
- Public URL serving natural-language Q&A grounded in expense data - citations rendered, refusals honest
- Streaming responses with safe partial rendering, functional cancel, and accessible updates
- Latency budget enforced server-side; over-budget streams terminate with partial answer plus honest note
- Cost budgets enforced: per-request token ceiling and per-period cap triggering degradation before rejection
- AI observability live: sanitized per-request logs capturing tokens, latencies and computed costs; dashboard queryable
- Kill switch tested LIVE in production flipping to keyword-search fallback with honest labeling
- Pre-ship gates green: eval suite, red-team pack; post-deploy smoke script executed; demo access documented for visitors
Context for your agent
Phase-gate my finale build. Phase 1 brief: feature scope (natural-language expense Q&A via my RAG stack), latency target, cost ceilings, degradation ladder - I approve numbers before code. Phase 2 build: streaming endpoint with safe partial rendering and cancel, budget enforcement, sanitized observability wired to my dashboards, kill switch to keyword fallback. Phase 3 gates: eval + red-team suites green. Phase 4 ship: deploy rehearsal, LIVE kill-switch test, production smoke script, demo-access docs. Hold me to every gate - refuse skips. Non-goals: voice input, multi-tenant anything, autonomous actions.
Verify it - twice
Check every item yourself first. Then hand your agent the prompt and compare verdicts.
Verification prompt - paste into your AI agent
Conduct the final acceptance review of Ledger Copilot as release gatekeeper - the last dual-check of the curriculum. Work through: (1) exercise the live feature: grounded answer with citations, honest refusal, streamed rendering, cancel mid-stream, (2) force a budget breach and observe degradation ladder behavior against documented policy, (3) flip the kill switch yourself and confirm fallback serves with honest labeling, then restore, (4) inspect sanitized logs for privacy leaks and completeness of token/latency/cost capture, (5) rerun eval and attack suites against PRODUCTION configuration, reporting scores. Finish with SHIP or DO-NOT-SHIP verdict and, if the latter, the exact blocking list. Then tell me: which single skill from Levels 0-4 carried the most weight in this build?
Run this against “Ship: AI-Native Product” after your own checklist pass. Where the agent and you disagree, investigate before moving on.
Review questions
Answer out loud. If you cannot, revisit before moving on.
Recite your launch checklist and which gates almost got skipped under excitement.
Trace one production question: tokens, cost, latency, cache status - from logs.
The curriculum asked you to direct agents, verify independently and gate releases - where did that discipline save THIS build?
You can now
- Ship AI features publicly with streaming UX and enforced budgets
- Observe AI behavior per-request: costs, latencies, sanitized forensics
- Gate AI releases on evals, red-teams, smokes and rehearsed kill switches