Public bake-off · classify

Best small model for Classify support requests as access or billing

Winner: cross-encoder/nli-deberta-v3-xsmall incumbent

challenger j-hartmann/emotion-english-distilroberta-base was already resident and competed with 3 family incumbents. The incumbent won.

Bake-off bo_f38871337530717bdd2bb1f8c2b9f4be · 2026-08-18T05:39:54.104071Z

Are you the winning model’s author?

Add this exact line to the winner’s public Hugging Face README.md, then call the claim endpoint with its repo id.

Bay-Run-Claim: bo_f38871337530717bdd2bb1f8c2b9f4be · sha256:a578076113d4cb2e76bf041198163c0dad95dd37adb4df2f17f63ad70e93118a

Scorecard

Rank	Model	Metrics	Predicted vs expected	Latency	Status
1	`cross-encoder/nli-deberta-v3-xsmall`incumbentwinner	accuracy 1.0macro_f1 1.0n 3	Not retained	692 ms	scored
2	`distilbert/distilbert-base-uncased-finetuned-sst-2-english`incumbent	accuracy 0.0macro_f1 0.0n 3	Not retained	99 ms	scored
3	`cardiffnlp/twitter-roberta-base-sentiment-latest`incumbent	accuracy 0.0macro_f1 0.0n 3	Not retained	11370 ms	scored
4	`j-hartmann/emotion-english-distilroberta-base`challenger	accuracy 0.0macro_f1 0.0n 3	Not retained	10017 ms	scored

Signed receipt

This result is bound to its task, candidates, scorecard, and winner.

sha256:a578076113d4cb2e76bf041198163c0dad95dd37adb4df2f17f63ad70e93118a

Winner badge

Copy-paste Markdown:

[![Bay Run bake-off winner · receipted](https://run.huggingbay.xyz/bakeoffs/bo_f38871337530717bdd2bb1f8c2b9f4be/badge.svg)](https://run.huggingbay.xyz/bakeoffs/bo_f38871337530717bdd2bb1f8c2b9f4be)

Reproduce it

The first bake-off is anonymous. Run the same labeled request directly:

curl -sS https://run.huggingbay.xyz/v1/bakeoff -H 'Content-Type: application/json' --data '{"task_description":"Classify support requests as access or billing","examples":[{"input":"Cannot sign in","expected":"access"},{"input":"Refund my invoice","expected":"billing"},{"input":"Login still fails","expected":"access"}],"challenger_model":"j-hartmann/emotion-english-distilroberta-base"}'