Public bake-off · classify
Weak field for Classify support tickets as access_issue, billing_issue, delivery_issue, account_update, or fulfillment_issue
Top measured: cross-encoder/nli-deberta-v3-xsmall incumbent
Weak field. The top measured accuracy did not clear the public 0.60/chance gate. This page makes no best-model, badge, or author claim.
Bake-off bo_a46e3dff6c59ddc75449251aa3851fdd ·
Scorecard
| Rank | Model | Metrics | Predicted vs expected | Latency | Status |
|---|---|---|---|---|---|
| 1 | cross-encoder/nli-deberta-v3-xsmallincumbent | accuracy 0.4macro_f1 0.4n 5 | Not retained | 1264 ms | scored |
| 2 | distilbert/distilbert-base-uncased-finetuned-sst-2-englishincumbent | accuracy 0.0macro_f1 0.0n 5 | Not retained | 161 ms | scored |
| 3 | cardiffnlp/twitter-roberta-base-sentiment-latestincumbent | accuracy 0.0macro_f1 0.0n 5 | Not retained | 11962 ms | scored |
Signed receipt
This result is bound to its task, candidates, scorecard, and winner.
sha256:f9aeb50cd4684f32b7f8488bdffeedfb8b5b5334d3822a691020a8ab5278023c
Reproduce it
The first bake-off is anonymous. Run the same labeled request directly:
curl -sS https://run.huggingbay.xyz/v1/bakeoff -H 'Content-Type: application/json' --data '{"task_description":"Classify support tickets as access_issue, billing_issue, delivery_issue, account_update, or fulfillment_issue","examples":[{"input":"Cannot sign in","expected":"access_issue"},{"input":"Refund the invoice","expected":"billing_issue"},{"input":"Shipment is late","expected":"delivery_issue"},{"input":"Setup is complete","expected":"account_update"},{"input":"Order is damaged","expected":"fulfillment_issue"}]}'