Earlier this month, an OpenAI engineer announced he was hiring for a new team.
OpenAI is building a new team called "Applied Evals," which helps businesses refine complex processes using AI, like refund requests or migrating code. The team will also focus on voice AI and multi-step reasoning tasks, said Shyamal Anadkat, who is leading that team at OpenAI. Pay ranges from $255,000 to $325,000 for the role plus equity, the new listing said.
Evals refer to measuring the capabilities of AI models, which Anadkat called "the most critical part of actually building AI products."
This new team reflects a shift across the AI talent landscape.. The tech industry has long hired technical minds to build and train AI models, and now people with specific subject-matter expertise are being tapped to make these models useful, AI founders and executives told Business Insider.
Anadkat's post said his team will comprise engineers with "real, deep, lived expertise." He said it will start with generalists. He plans to bring in specialists from different fields, such as software engineers for code-related tasks and, perhaps eventually, humanities-focused workers if there is more demand for tasks like writing.
Anadkat told Business Insider the team will work with companies using OpenAI's developer platform to help them create their own evals for specific use cases.
"OpenAI started as a research lab, and then we built a product," he said. "Now the shift that, at least I hope we'll see, is how research and product can work very closely together and collaborate on some of this to basically define what good looks like."
Anadkat's team is starting small and will grow with demand. It's exclusively focused on business customers, operating separately from the teams working on OpenAI's consumer-facing businesses, like apps or consulting.
Applied Evals will work with OpenAI's sales and business teams to determine what projects to prioritize based on customer need and where its models are underperforming, Anadkat said.
The AI talent shift
The AI industry has long had engineers work on evals within companies, large and small, though they mostly evaluated if the AI model was working well or not.
Over time, they've shifted from "thumbs up or down" to being more focused on context and asking the right questions, said Michael Jacobides, a strategy and entrepreneurship professor at London Business School.
"There's probably a hundred people in the world that could lead a team to build these frontier models," said Justin Farris, vice president of product at Read AI, "but there's so much work that needs to be done to take those and make them useful."
AI is increasingly moving from generic to specific use cases, said Tanmai Gopal, CEO of PromptQL.
"For a lot of applied use cases, the ways of working out what good is and what bad is start to become quite nuanced," he said.
Have a tip? Contact this reporter via email at gweiss@bjinnox.com or Signal at @geoffweiss.25. Use a personal email address and a nonwork device; here's our guide to sharing information securely.
