Why I Built It
Most model comparisons are shallow. I needed something that could compare models against real prompts, track the output quality, and keep a record of why one choice won over another.
What It Needs To Do
Benchmark needs to route tasks, capture outputs, score results, and make the winning choice obvious. It also needs to keep enough data around that I can defend why a model was chosen for a production job.
Want the full architecture?
If you want to see how Benchmark plugs into Alfred and model routing, I can walk you through the full stack.
Book a Call