Benchmark researcher_
Remote or New York · full-time
What you'll do
Decide how the bench will score: which datasets qualify, how tasks are isolated, how contamination is detected, what "found" means, how many repetitions a cell needs and how the mean is reported. Stand up the first table, then answer disputes in public and re-run when they are upheld. Write the reports that publish the results.
What we look for
You have built evaluations for machine-learning systems or security tooling and been burned by leakage at least once. Statistics you can defend to a hostile reader. A habit of publishing negative results.
What we offer
The most public job in the company: every number the platform will stand behind is yours to get right, and the methodology page carries your name.