@agentskit/eval — for agents

Evaluation harness + deterministic replay + snapshot testing + diff + CI reporters.

#Install

npm install @agentskit/eval

#Primary exports

runEval({ agent, suite }) — run an EvalSuite against any async agent fn.

#Subpaths

Subpath	Contents
`@agentskit/eval/replay`	`createRecordingAdapter`, `createReplayAdapter`, cassettes, `createTimeTravelSession`, `replayAgainst`, `summarizeReplay`. See Deterministic replay, Time travel, Replay-different-model.
`@agentskit/eval/snapshot`	`matchPromptSnapshot` (exact / normalized / similarity). See Snapshots.
`@agentskit/eval/diff`	`promptDiff`, `attributePromptChange`, `formatDiff`. See Prompt diff.
`@agentskit/eval/ci`	`renderJUnit`, `renderMarkdown`, `renderGitHubAnnotations`, `reportToCi`. See Evals in CI.

#Minimal example

import { runEval } from '@agentskit/eval'

const result = await runEval({
  agent: async (input) => (await runtime.run(input)).content,
  suite: {
    name: 'qa',
    cases: [{ input: 'Capital of France?', expected: 'Paris' }],
  },
})

console.log(`${result.passed}/${result.totalCases}`)

@agentskit/eval-braintrust — Braintrust reporter backend.
@agentskit/runtime.
@agentskit/observability — trace + cost data feeds eval comparisons.
@agentskit/core/eval-format — portable format spec.

#Source

Explore nearby

✎ Edit this page on GitHub·Found a problem? Open an issue →·How to contribute →

@agentskit/eval — for agents

#Install

#Primary exports

#Subpaths

#Minimal example

#Related

#Source

Explore nearby

On this page