Skip to content
build-3000
ai

Ship: AI-Native Product

The Level 5 finale - streaming UX, cost/latency budgets, AI observability, kill switches - shipped publicly.

Before this: AI Security and Evaluation

Objective

Combine every Level 5 skill into one public delivery: a grounded natural-language feature on your Ledger with streaming responses, enforced budgets, per-request AI observability, graceful degradation and a kill switch - evaluated pre-ship and smoke-tested post-deploy.

Vocabulary

The words to use when directing your agent.

streaming response
Delivering completion tokens progressively so users watch text arrive instead of staring at spinners.
latency budget
The maximum acceptable wait per interaction, enforced with timeouts and stream deadlines.
cost budget
Per-request and per-period ceilings on token spend, enforced in code.
AI observability
Per-request logs of prompts (sanitized), completions, token counts, latencies and computed costs.
kill switch
Environment-level instant disable of the AI path, falling back to non-AI behavior without redeploy.

Understand it

The streaming UX contract

Streams transform perceived performance: first tokens arriving within moments beat perfect answers arriving after dead air. Contract details matter: render partial text safely (incomplete markdown included), keep cancel buttons functional mid-stream, handle cut-off streams gracefully, and preserve accessibility - screen readers need considered aria-live treatment, not raw token soup. Perceived speed is a design achievement, not an accident.

Budgets, degradation and the launch checklist

Latency budget: deadline enforced server-side; streams that exceed it terminate with partial-plus-honest-note. Cost budget: per-request token ceilings and per-period aggregate caps; approaching limits triggers degradation (smaller contexts, cached answers) BEFORE hard rejection. Pre-launch: eval suites green, red-team pass current, kill switch tested LIVE, dashboards watching SLOs. Post-launch: smoke scripts exercising the AI path in production, rollback rehearsed - your Level 3 rhythms, now with GPUs somewhere behind them.

Core concepts

AI features fail differently - design those failures

Traditional features mostly succeed instantly or error clearly. AI features add slow-successes, confident-wrongs and capacity-dependent availability. Shipped AI products design FOR this: streams starting fast so waits feel shorter, timeouts bounding worst cases, degraded modes (cached answers, keyword fallback) keeping features useful, and honest labels when answers run on reduced quality.

Every generation is a bill and a log line

Production AI means per-request economics and forensics: token counts logged, costs computed from configured rates, latencies tracked, prompts sanitized before logging. Budgets ENFORCE in code - reject or degrade before exceeding ceilings, not after the invoice arrives. What you cannot see, you cannot cap.

Examples

Answers that stream, budgets that bite

'How much did I spend on dining last month?' starts rendering within moments, cites its source rows, finishes comfortably inside deadline. Behind scenes: token counts logged, cost computed, deadline armed, cache primed for repeat asks. Second identical question serves from cache - flagged as such - costing approximately nothing.

The flip that saved the evening

Provider incident degrades answer quality during peak hours. Kill switch flips the feature to keyword-search fallback: slightly dumber, fully honest labeling, zero downtime drama. Postmortem next morning uses per-request logs to quantify impact precisely. Ungated deployments would have spent the evening apologizing on social media instead.

Common mistakes

Common mistake

Shipping the AI path without budgets because 'usage will be small at first'.

Fix

Small usage grows or goes viral unexpectedly. Enforce ceilings from commit one - retrofitting budgets after an invoice shock is triage, not engineering.

Common mistake

Logging full prompts containing user data, or logging nothing at all.

Fix

Sanitize before persisting: strip personal fields, hash identifiers, retain diagnostic structure. Both extremes - forensic drought and privacy flood - are incidents waiting for calendars.

Working with your AI agent

Phase the engagement like your Level 2 capstone: brief and budgets agreed FIRST (latency targets, cost ceilings, degradation ladder), then build, then evaluate (eval + red-team suites green), then deploy rehearsal with kill-switch test, then production smoke. Demand completion evidence at every phase gate - this finale certifies the whole curriculum's working style, AI-flavored.

Build it

The module’s real project. Your agent builds; you direct and inspect.

Project: Ledger Copilot, Shipped

The curriculum's final boss: ship Ledger Copilot publicly - natural-language questions answered from YOUR expense data via your RAG stack, streaming UI, enforced latency/cost budgets, sanitized AI observability, kill switch, evaluated pre-ship and smoke-tested live with demo access for visitors.

Requirements

  • Public URL serving natural-language Q&A grounded in expense data - citations rendered, refusals honest
  • Streaming responses with safe partial rendering, functional cancel, and accessible updates
  • Latency budget enforced server-side; over-budget streams terminate with partial answer plus honest note
  • Cost budgets enforced: per-request token ceiling and per-period cap triggering degradation before rejection
  • AI observability live: sanitized per-request logs capturing tokens, latencies and computed costs; dashboard queryable
  • Kill switch tested LIVE in production flipping to keyword-search fallback with honest labeling
  • Pre-ship gates green: eval suite, red-team pack; post-deploy smoke script executed; demo access documented for visitors

Context for your agent

Phase-gate my finale build. Phase 1 brief: feature scope (natural-language expense Q&A via my RAG stack), latency target, cost ceilings, degradation ladder - I approve numbers before code. Phase 2 build: streaming endpoint with safe partial rendering and cancel, budget enforcement, sanitized observability wired to my dashboards, kill switch to keyword fallback. Phase 3 gates: eval + red-team suites green. Phase 4 ship: deploy rehearsal, LIVE kill-switch test, production smoke script, demo-access docs. Hold me to every gate - refuse skips. Non-goals: voice input, multi-tenant anything, autonomous actions.

Verify it - twice

Check every item yourself first. Then hand your agent the prompt and compare verdicts.

Self-verified0/13
Functional
Architecture
Security
Testing
Production

Verification prompt - paste into your AI agent

Conduct the final acceptance review of Ledger Copilot as release gatekeeper - the last dual-check of the curriculum. Work through: (1) exercise the live feature: grounded answer with citations, honest refusal, streamed rendering, cancel mid-stream, (2) force a budget breach and observe degradation ladder behavior against documented policy, (3) flip the kill switch yourself and confirm fallback serves with honest labeling, then restore, (4) inspect sanitized logs for privacy leaks and completeness of token/latency/cost capture, (5) rerun eval and attack suites against PRODUCTION configuration, reporting scores. Finish with SHIP or DO-NOT-SHIP verdict and, if the latter, the exact blocking list. Then tell me: which single skill from Levels 0-4 carried the most weight in this build?

Run this against “Ship: AI-Native Product” after your own checklist pass. Where the agent and you disagree, investigate before moving on.

Review questions

Answer out loud. If you cannot, revisit before moving on.

  1. Recite your launch checklist and which gates almost got skipped under excitement.

  2. Trace one production question: tokens, cost, latency, cache status - from logs.

  3. The curriculum asked you to direct agents, verify independently and gate releases - where did that discipline save THIS build?

You can now

  • Ship AI features publicly with streaming UX and enforced budgets
  • Observe AI behavior per-request: costs, latencies, sanitized forensics
  • Gate AI releases on evals, red-teams, smokes and rehearsed kill switches