OSS-first docs

These docs teach the open system first: contracts, generated surfaces, runtimes, governance, and incremental adoption. Studio shows up as the operating layer on top, not as the source of truth.

AI index

Auto-Evolution Runbook

Configure sampling, approvals, experiments, and golden tests so your app keeps improving without regressions.

Pipeline overview

  1. Tracing middleware emits telemetry samples with tenant/user context.

  2. IntentAggregator batches samples (15m window, configurable).

  3. IntentDetector raises error/latency/throughput signals.

  4. SpecGenerator produces proposals with confidence scores.

  5. ApprovalWorkflow routes low-confidence suggestions to humans.

  6. SpecExperimentRegistry rolls out approved variants using guardrails.

  7. TrafficRecorder captures new traffic and GoldenTestGenerator outputs suites.

Key environment variables

# Telemetry sampling
AUTO_EVOLUTION_SAMPLE_RATE=0.02
AUTO_EVOLUTION_WINDOW_MS=900000

# Suggestion thresholds
AUTO_EVOLUTION_MIN_CONFIDENCE=0.55
AUTO_EVOLUTION_AUTO_APPROVE=0.2

# Guardrails (fallback defaults)
AUTO_EVOLUTION_MAX_ERROR_RATE=0.02
AUTO_EVOLUTION_MAX_P99_MS=500

Operations checklist

  • Review new `SpecSuggestion` records every morning (UI coming soon; Prisma view today).

  • Keep at least one reviewer per domain on-call for approvals.

  • Attach `SpecExperimentAdapter.trackOutcome` to every runtime adapter.

  • Schedule `contractspec test generate` nightly (or on deploy) to refresh golden suites.

  • Feed `runGoldenTests` into CI to gate merges touching evolved specs.

Sampling

Start at 1–2% of traffic. Increase only after sanitization is verified.

Approvals

High-risk ops (money, compliance) should never auto-approve. Use ApprovalWorkflow notes for audit.

On-call alerts

Tie SpecExperimentController rollbacks into PagerDuty/Slack so engineers see guardrail trips immediately.