ContractSpec docs

OSS-first docs

These docs teach the open system first: contracts, generated surfaces, runtimes, governance, and incremental adoption. Studio shows up as the operating layer on top, not as the source of truth.

AI index

Eval runner — PASS / PARTIAL / FAIL diff

Runs EvalFixture assertions against actual LLM output, produces PASS/PARTIAL/FAIL with budget violation list.

field.key.label
Eval runner — PASS / PARTIAL / FAIL diff
field.version.label
field.type.label
field.title.label
Eval runner — PASS / PARTIAL / FAIL diff
field.description.label

Runs EvalFixture assertions against actual LLM output, produces PASS/PARTIAL/FAIL with budget violation list.

field.tags.label
agent,eval,graphOps,llm,m5
field.owners.label
field.stability.label
internal

Diffs actual tool output against fixture assertions and emits a structured EvalResult.

Status rules: FAIL — any assertion with tolerance=0.0 fails PARTIAL — no zero-tolerance failures; at least one assertion with 0 < tolerance < 0.5 fails PASS — all low-tolerance assertions pass (permissive ≥0.5 failures ignored)

Budget violations are non-fatal annotations (do not change status). Use checkSessionBudget() to enforce the $0.50 / 50-query session ceiling.