Public bake-off · classify
Weak field for Classify support tickets by workflow
Top measured: cross-encoder/nli-deberta-v3-xsmall incumbent
Weak field. The top measured accuracy did not clear the public 0.60/chance gate. This page makes no best-model, badge, or author claim.
Bake-off bo_de9f3884028437c7f0e848d249588136 ·
Scorecard
| Rank | Model | Metrics | Predicted vs expected | Latency | Status |
|---|---|---|---|---|---|
| 1 | cross-encoder/nli-deberta-v3-xsmallincumbent | accuracy 0.4macro_f1 0.4n 5 | Not retained | 1200 ms | scored |
| 2 | distilbert/distilbert-base-uncased-finetuned-sst-2-englishincumbent | accuracy 0.0macro_f1 0.0n 5 | Not retained | 535 ms | scored |
| 3 | cardiffnlp/twitter-roberta-base-sentiment-latestincumbent | accuracy 0.0macro_f1 0.0n 5 | Not retained | 12643 ms | scored |
Signed receipt
This result is bound to its task, candidates, scorecard, and winner.
sha256:eaccc5177ecbbe7e45626c50ce31e14dd27680403b6be63dff7f2d8722640c87
Reproduce it
The first bake-off is anonymous. Run the same labeled request directly:
curl -sS https://run.huggingbay.xyz/v1/bakeoff -H 'Content-Type: application/json' --data '{"task_description":"Classify support tickets by workflow","examples":[{"input":"Cannot sign in","expected":"access_issue"},{"input":"Refund the invoice","expected":"billing_issue"},{"input":"Shipment is late","expected":"delivery_issue"},{"input":"Setup is complete","expected":"account_update"},{"input":"Order is damaged","expected":"fulfillment_issue"}]}'