agent-evaluation

Evidence-first, crash-resumable self-evolution engine for DeepSeek Harness and Harbor.

Paired A/B evaluation infrastructure for DeepSeek Harness (dsh) components: interleaved repeated runs through the real runtime, verifier self-checks, one-variable arms, regression gating, cache- and calendar-aware cost, CLI + web UI

DeepSeek Harness plugin that validates and ranks 3/5 independent coding-agent patches before approval-gated apply.

Agent evaluation platform for DeepSeek Harness: benchmark YAML, headless dsh orchestration, trace-based metrics, LLM judge, paired A/B, keyless replay, and cross-harness import.

Agent eval framework over dsh headless runs: case runner, session-trace assertions, and a scripted mock-LLM layer for plugin intent tests.

A blind, fair, local DSH Web arena: same task, isolated worktrees, shared verification, judge before reveal.

Compare multiple coding models side by side in DeepSeek Harness with isolated Git worktrees, live traces, and adoptable results

Planned repeatable agent and plugin regression evaluation for DeepSeek Harness

DSH-native multi-runtime baseline, ablation, and reproducible evaluation control plane

Reproducible paired A/B experiments for DeepSeek Harness plugins, with isolated runners, auditable evidence, and deterministic promotion gates.

Install with npm i dsh-benchup. Reproducible, profile-aware benchmarks for DeepSeek Harness — compare models, plugins, prompts, and agent strategies.

Offline evaluation and guarded self-evolution loop for DeepSeek Harness