Public bake-off · rerank

Best small model for Rerank passages by relevance to the query

Winner: cross-encoder/ms-marco-MiniLM-L4-v2 incumbent

Resident family incumbents were compared on the submitted labels.

Bake-off bo_6b50ebbb2090f47f5fa9ebf66dc891f4 · 2026-08-19T21:46:45.700153Z

Are you the winning model’s author?

Add this exact line to the winner’s public Hugging Face README.md, then call the claim endpoint with its repo id.

Bay-Run-Claim: bo_6b50ebbb2090f47f5fa9ebf66dc891f4 · sha256:afa3253cfa8047871821afb00e4dcddfc618f1f3ba18cd947796a36dc97b0a63

Scorecard

Rank	Model	Metrics	Predicted vs expected	Latency	Status
1	`cross-encoder/ms-marco-MiniLM-L4-v2`incumbentwinner	hit@5 1.0mrr 0.8333ndcg@5 0.877	Not retained	89 ms	scored
2	`cross-encoder/ms-marco-TinyBERT-L2-v2`incumbent	hit@5 1.0mrr 0.6667ndcg@5 0.754	Not retained	62 ms	scored

Signed receipt

This result is bound to its task, candidates, scorecard, and winner.

sha256:afa3253cfa8047871821afb00e4dcddfc618f1f3ba18cd947796a36dc97b0a63

Winner badge

Copy-paste Markdown:

[![Bay Run bake-off winner · receipted](https://run.huggingbay.xyz/bakeoffs/bo_6b50ebbb2090f47f5fa9ebf66dc891f4/badge.svg)](https://run.huggingbay.xyz/bakeoffs/bo_6b50ebbb2090f47f5fa9ebf66dc891f4)

Reproduce it

The first bake-off is anonymous. Run the same labeled request directly:

curl -sS https://run.huggingbay.xyz/v1/bakeoff -H 'Content-Type: application/json' --data '{"task_description":"Rerank passages by relevance to the query","examples":[{"input":"password reset || Reset your password from account settings.","expected":"relevant"},{"input":"password reset || Your invoice PDF is in billing.","expected":"irrelevant"},{"input":"password reset || Click forgot password on the login page.","expected":"relevant"}]}'