ContractSpec docs

OSS-first docs

These docs teach the open system first: contracts, generated surfaces, runtimes, governance, and incremental adoption. Studio shows up as the operating layer on top, not as the source of truth.

platform.provider-ranking

Benchmark ingestion, custom benchmark runs, and AI model ranking with leaderboards

  • Type: feature
  • Version: 1.0.0
  • Tags: ai, ranking, benchmark
  • File: packages/libs/contracts-spec/src/provider-ranking/provider-ranking.feature.ts
  • field.key.label
    platform.provider-ranking
    field.version.label
    1.0.0
    field.type.label
    feature
    field.title.label
    platform.provider-ranking
    field.description.label

    Benchmark ingestion, custom benchmark runs, and AI model ranking with leaderboards

  • Type: feature
  • Version: 1.0.0
  • Tags: ai, ranking, benchmark
  • File: packages/libs/contracts-spec/src/provider-ranking/provider-ranking.feature.ts
  • field.tags.label
    ai,ranking,benchmark
    field.owners.label
    field.stability.label

    Benchmark ingestion, custom benchmark runs, and AI model ranking with leaderboards

    Operations (6)

    `provider-ranking.benchmark.ingest` (v1.0.0)

    `provider-ranking.benchmark.run-custom` (v1.0.0)

    `provider-ranking.ranking.refresh` (v1.0.0)

    `provider-ranking.ranking.get` (v1.0.0)

    `provider-ranking.benchmark.results.list` (v1.0.0)

    `provider-ranking.model.profile.get` (v1.0.0)

    Events (3)

    `provider-ranking.benchmark.ingested` (v1.0.0)

    `provider-ranking.benchmark.custom.completed` (v1.0.0)

    `provider-ranking.ranking.updated` (v1.0.0)

    Presentations (1)

    `provider-ranking.model.comparison` (v1.0.0)

    Source Definition

    import type { DocBlock } from '../docs/types';
    import { defineFeature } from '../features';
    import {
    	PROVIDER_RANKING_DOMAIN,
    	PROVIDER_RANKING_OWNERS,
    	PROVIDER_RANKING_STABILITY,
    	PROVIDER_RANKING_TAGS,
    } from './constants';
    
    export const ProviderRankingSystemDocBlock = {
    	id: 'docs.tech.provider-ranking.system',
    	title: 'AI Provider Ranking',
    	summary:
    		'Benchmark ingestion, scoring, and ranking for AI providers and models.',
    	kind: 'reference',
    	visibility: 'public',
    	route: '/docs/tech/provider-ranking/system',
    	tags: ['ai', 'ranking', 'benchmark'],
    	body: `# AI Provider Ranking
    
    Defines the core operations, events, and UI surfaces for ingesting benchmark data, computing composite rankings, and comparing AI models across dimensions.
    `,
    } satisfies DocBlock;
    
    export const ProviderRankingFeature = defineFeature({
    	meta: {
    		key: 'platform.provider-ranking',
    		version: '1.0.0',
    		title: 'Provider Ranking',
    		description:
    			'Benchmark ingestion, custom benchmark runs, and AI model ranking with leaderboards',
    		domain: PROVIDER_RANKING_DOMAIN,
    		owners: PROVIDER_RANKING_OWNERS,
    		tags: [...PROVIDER_RANKING_TAGS],
    		stability: PROVIDER_RANKING_STABILITY,
    	},
    
    	operations: [
    		{ key: 'provider-ranking.benchmark.ingest', version: '1.0.0' },
    		{ key: 'provider-ranking.benchmark.run-custom', version: '1.0.0' },
    		{ key: 'provider-ranking.ranking.refresh', version: '1.0.0' },
    		{ key: 'provider-ranking.ranking.get', version: '1.0.0' },
    		{ key: 'provider-ranking.benchmark.results.list', version: '1.0.0' },
    		{ key: 'provider-ranking.model.profile.get', version: '1.0.0' },
    	],
    
    	events: [
    		{ key: 'provider-ranking.benchmark.ingested', version: '1.0.0' },
    		{ key: 'provider-ranking.benchmark.custom.completed', version: '1.0.0' },
    		{ key: 'provider-ranking.ranking.updated', version: '1.0.0' },
    	],
    
    	presentations: [
    		{ key: 'provider-ranking.model.comparison', version: '1.0.0' },
    	],
    
    	capabilities: {
    		provides: [{ key: 'provider-ranking.system', version: '1.0.0' }],
    	},
    
    	dataViews: [
    		{ key: 'provider-ranking.ranking.index', version: '1.0.0' },
    		{ key: 'provider-ranking.benchmark.results.index', version: '1.0.0' },
    	],
    
    	forms: [
    		{ key: 'provider-ranking.benchmark.ingest.form', version: '1.0.0' },
    		{ key: 'provider-ranking.benchmark.run-custom.form', version: '1.0.0' },
    	],
    });