Swebench

“But the key benchmark that they took was this benchmark called SWE-bench. And people here might have heard of SWE-bench, but the idea here this is-- it's 500 problems.”

Amjad Masad

youtube · 2025-08-11

Recommended

“There's a benchmark called SWEBench software engineering bench and you start seeing the score going up dramatically. I don't know I think we were at like 10% last year and now we're at like 70% and 80%.”