Public bake-off · classify

Best small model for Classify each support ticket into billing, technical, or account category

Winner: cross-encoder/nli-deberta-v3-xsmall incumbent

Resident family incumbents were compared on the submitted labels.

Bake-off bo_ebaea43dc8c2ffd83100bf60f4ebcb87 · 2026-08-19T19:34:39.817740Z

Are you the winning model’s author?

Add this exact line to the winner’s public Hugging Face README.md, then call the claim endpoint with its repo id.

Bay-Run-Claim: bo_ebaea43dc8c2ffd83100bf60f4ebcb87 · sha256:1daeded0078a8b15011587498b81b29529e2867c7cfc0a3e89074462475463bb

Scorecard

Rank	Model	Metrics	Predicted vs expected	Latency	Status
1	`cross-encoder/nli-deberta-v3-xsmall`incumbentwinner	accuracy 1.0macro_f1 1.0n 4	match `billing` vs `billing` match `technical` vs `technical` match `account` vs `account` match `billing` vs `billing`	1314 ms	scored
2	`typeform/distilbert-base-uncased-mnli`incumbent	accuracy 0.5macro_f1 0.3889n 4	match `billing` vs `billing` miss `account` vs `technical` match `account` vs `account` miss `account` vs `billing`	1073 ms	scored

Signed receipt

This result is bound to its task, candidates, scorecard, and winner.

sha256:1daeded0078a8b15011587498b81b29529e2867c7cfc0a3e89074462475463bb

Winner badge

Copy-paste Markdown:

[![Bay Run bake-off winner · receipted](https://run.huggingbay.xyz/bakeoffs/bo_ebaea43dc8c2ffd83100bf60f4ebcb87/badge.svg)](https://run.huggingbay.xyz/bakeoffs/bo_ebaea43dc8c2ffd83100bf60f4ebcb87)

Reproduce it

The first bake-off is anonymous. Run the same labeled request directly:

curl -sS https://run.huggingbay.xyz/v1/bakeoff -H 'Content-Type: application/json' --data '{"task_description":"Classify each support ticket into billing, technical, or account category","examples":[{"input":"I was charged twice on my invoice","expected":"billing"},{"input":"The app crashes when I click export","expected":"technical"},{"input":"Please update my email address","expected":"account"},{"input":"Refund my duplicate payment","expected":"billing"}]}'