OSS-first docs

These docs teach the open system first: contracts, generated surfaces, runtimes, governance, and incremental adoption. Studio shows up as the operating layer on top, not as the source of truth.

AI index

harness.evaluation.run

Run a harness scenario or suite as an evaluation.

  • Type: operation (command)
  • Version: 1.0.0
  • Tags: evaluation, run
  • File: packages/libs/contracts-spec/src/harness/commands/harnessEvaluationRun.command.ts
    • field.key.label
    harness.evaluation.run
    field.version.label
    1.0.0
    field.type.label
    operation (command)
    field.title.label
    harness.evaluation.run
    field.description.label

    Run a harness scenario or suite as an evaluation.

  • Type: operation (command)
  • Version: 1.0.0
  • Tags: evaluation, run
  • File: packages/libs/contracts-spec/src/harness/commands/harnessEvaluationRun.command.ts
    • field.tags.label
    evaluation,run
    field.owners.label
    field.stability.label

    Run a harness scenario or suite as an evaluation.

    Goal

    Produce assertion-backed evidence for QA and regression workflows.

    Context

    Used by evaluation pipelines and benchmark suites.

    Source Definition

    import { ScalarTypeEnum, SchemaModel } from '@lssm-tech/lib.schema';
import { defineCommand } from '../../operations';
import {
	HARNESS_DOMAIN,
	HARNESS_OWNERS,
	HARNESS_STABILITY,
	HARNESS_TAGS,
} from '../constants';
import { HarnessEvaluationCompletedEvent } from '../events/harnessEvaluationCompleted.event';
import { HarnessEvaluationModel } from '../models';

export const HarnessEvaluationRunCommand = defineCommand({
	meta: {
		key: 'harness.evaluation.run',
		title: 'Run Harness Evaluation',
		version: '1.0.0',
		description: 'Run a harness scenario or suite as an evaluation.',
		goal: 'Produce assertion-backed evidence for QA and regression workflows.',
		context: 'Used by evaluation pipelines and benchmark suites.',
		domain: HARNESS_DOMAIN,
		owners: HARNESS_OWNERS,
		tags: [...HARNESS_TAGS, 'evaluation', 'run'],
		stability: HARNESS_STABILITY,
	},
	capability: { key: 'harness.evaluation', version: '1.0.0' },
	io: {
		input: HarnessEvaluationRunInput,
		output: HarnessEvaluationRunOutput,
	},
	policy: {
		auth: 'user',
		pii: [],
	},
	sideEffects: {
		emits: [
			{
				ref: HarnessEvaluationCompletedEvent.meta,
				when: 'Evaluation execution finishes and assertion results are persisted.',
			},
		],
	},
});