“[I]n recent years, companies have coalesced around using the term ‘AI’ in these kinds of workloads (and in their related marketing),” Primate Labs says of the name change. “To ensure that everyone, from engineers to performance enthusiasts, understands what this benchmark does and how it works, we felt it was time for an update.”
Earlier this week, ChatGPT-maker OpenAI announced a new version of its own AI model benchmark. SWE-bench Verified is a “human-validated” offering that uses human validation to determine models’ efficacy in solving “real-world issues.”
Content Courtesy – Tech Crunch
