# Evalset Desk > Paste the export of one LLM eval run - inputs, expected answers, the model's outputs and, optionally, a baseline's outputs, judge scores, human ratings, slices and retrieved ids - as CSV, TSV, JSONL, a JSON array or a Markdown table. The browser scores every row, puts a 95% interval on every rate, pairs the candidate against the baseline (wins, regressions, an exact sign test), measures Cohen's kappa between raters or between the judge and a human, splits the result by slice, and checks a stated claim and your ship bar against the rows, for free. Then a verdict lane says ship, hold or fix the eval first, and a judge lane designs the LLM-as-judge for the feature. URL: https://evalset-desk.skillsafe.ai/ API tutorial: https://evalset-desk.skillsafe.ai/api.html ## Who it is for Engineers, ML and product leads who changed a prompt, a model or a retrieval setup, ran it over an eval set, and must decide whether the new version ships - or who suspect the eval itself (a lenient LLM judge, token-overlap metrics on long answers, too few cases) cannot answer that. ## The free scoring (in the browser, no model) - Columns mapped by name: input (prompt, question, ticket ...), expected (reference, gold, label, answer ...), output (prediction, response, candidate ...), baseline (previous, control, output_a ...), score (judge, grade, pass ...), baseline_score, rater_1 and rater_2 (human ...), slice (category, segment, topic, language ...), retrieved and relevant ids. - Run types: classification (accuracy, per-class precision, recall and F1, macro F1, the majority-class baseline, the confusion pairs); text against references (match rate - exact or token F1 of 0.5 or more - exact match, mean token F1, mean ROUGE-L); judge-scored (pass rate at 0.5 on 0-1, 4 on 1-5, 7 on 1-10, 70 on 0-100, mean score); retrieval (hit rate, MRR, recall@k, NDCG@k, k up to 5). - Every rate with a 95% Wilson interval; the baseline pairing with wins, regressions, ties, the change in points with its interval, and an exact two-sided sign test on the discordant rows; Cohen's kappa between two raters and between the judge's pass/fail and a human's. - Findings (EV-01..): a claimed figure no metric gives or a "significant" claim the sign test does not support, a missed ship bar, regressions, judge-human kappa under 0.4 (high); empty outputs, rows with no reference or score, duplicate inputs, a change whose interval includes zero, fewer than 100 rows, a majority label of 60% or more, a slice 10 points below the whole, token overlap on references longer than 12 words, rater kappa under 0.6 (medium). - Exports: the scoring as Markdown (with a ship-bar line: clears it or not), every row as CSV (input, expected, output and baseline included), and the regressions and failing rows as their own CSV work list. ## The paid lanes (gpt-terra, metered) - verdict: ship, hold or fix the eval first (never ship while a high finding stands; fix the eval first when the judge does not track the humans); one answer per page finding, in order; 2-6 metrics read with their values quoted exactly; failure modes grouping the failing rows; every regression listed with what changed; gaps in the eval set with fixes; 1-5 next steps. - judge: the mode (pointwise, pairwise, reference-based or reference-free) the columns support; 3-6 criteria with weights summing to 100 and a pass and a fail test each; the complete judge prompt with its placeholders and a JSON reply; 4-8 calibration rows from the run with the grade a correct judge must give (the human's, when there is one); 3-6 new cases; 1-4 ship bars. After a verdict (the button under it), every failure mode it named gets a criterion. ## How replies are held to the input Every reply is reconciled in the browser against the page's scoring: a figure in the headline; every row, metric and finding id one the page sent; no figure the rows or the scoring do not contain (rounding allowed; proposed ship bars excepted); for a verdict, the rules above on the verdict, the counted findings confirmed, metric values identical to the page's, failure modes citing failing rows, the regressions exactly the page's; for a judge, the weights, the placeholders, every criterion named in the prompt, calibration matching the human rater or the label, and no new case repeating an existing input. ## Limits - The paid lanes see up to 48 rows (every regression first, then failing rows, then others; cells over 280 characters cut and marked) and the page's scoring of all rows, up to 5,000. - The page measures the run it is given; it cannot tell whether the references are right or whether the set looks like production traffic. - Token F1 and ROUGE-L are lexical: they underrate correct paraphrases, which is why the page flags them on long answers and the judge lane exists. ## Source Derived from the agent skill @wshobson/llm-evaluation (https://skillsafe.ai/skill/@wshobson/llm-evaluation), from the repository wshobson/agents (MIT): automated metrics for classification, text generation and retrieval; human evaluation with annotation guidelines and inter-rater agreement (Cohen's kappa); LLM-as-judge in pointwise, pairwise, reference-based and reference-free forms; A/B testing with statistical tests; regression detection; and the best practices and pitfalls of evaluating LLM applications. ## Pages - https://evalset-desk.skillsafe.ai/ - the app - https://evalset-desk.skillsafe.ai/api.html - API tutorial (cURL, Python, JavaScript, Go, Java, Ruby, PHP, C#)