Pair Rubric
Two model answers, one shared yes/no checklist. Atomic per-cell checks, virtualized to a thousand rubrics. The cleanest signal for SFT.
LabelHub · data production cockpit
Build labeling tasks, pre-review with AI, and export clean datasets.
One workspace for the complete flow: task publishing, visual form design, labeler workbench, AI review, human acceptance, and reproducible exports.
5
workspaces
0
trajectories
0
teaching signals
END-TO-END FLOW
1
Publish task
draft / open / paused
2
Import data
JSONL / CSV / Excel
3
Label workbench
autosave + validation
4
AI pre-review
scores + verdict
5
Human review
pass / send back
6
Export dataset
JSONL / CSV / Excel
OWNER
Task publishing without hidden setup
Create schema-driven tasks, import datasets, assign rows, and publish from one cockpit.
LABELER
A focused workbench for throughput
Claim items, autosave drafts, use field-level AI help, and see revision feedback.
REVIEWER
AI signal plus accountable human decisions
Review queues include AI scores, diff history, batch actions, and audit trails.
§ 02 TEMPLATES
Three modes. One engine. Pick the shape of the teaching.
Arena GSB
LMSYS-style head-to-head with multi-dimension 1–5 scoring. GSB verdict per dimension, plus required reasoning. Drives Bradley-Terry / Elo.
Agent Trace Eval
Score every step of an agent trajectory — tool calls, results, final answer. Per-step rubric + per-trajectory verdict. The flagship for agent eval.
Switch modes per task. Mix them per workspace. The rubric, the trust score, and the audit trail travel with you.