A familiar scenario is playing out all over corporate America: CEOs are meeting with their top tech executives and asking, "What are we doing with AI?"
So the tech exec goes in search of generative AI ideas to trial only to discover that training an AI model using the company's own data will cost millions in cloud computing costs. Plus, training isn't a one-time thing. AI models must be regularly fed.
Training an OpenAI-style model — known as a transformer model — takes lots of data, about six-times the number datapoints than the number of parameters the model will use to produce the asked-for result, Guido Appenzeller, former CTO of Intel's Data Center, said on an A16z podcast.
Since cloud computing fees are based on things like the amount of data uploaded and processed, companies spent an average of $4 million to train Chat GPT 3 apps, a Forrester analyst told CNBC last year, and more advanced models like GPT 4 could cost even more. AI models also require hefty, energy-consuming GPU chips. Researchers say that AI is already on track to consume as much energy as powering a small country.
"We can't keep going down this path where it's bigger, bigger, bigger, bigger, which is great for Nvidia, great to sell more GPUs, but at some point you have to start making it more efficient," says Christopher Savoie, founder and CEO of Zapata.
Zapata AI is an AI startup spun out of Harvard that uses quantum physics math to create AI models that need 10 times to 300 times less training data for equal or better accuracy, Zapata says.
The company, run by computing and quantum research scientists, has made a believer out of car racing company Andretti Global. After Andretti Global became a customer and used Zapata's AI technology to predict things like tire degradation, Mario and Michael Andretti's SPAC, Andretti Acquisition Corp., is taking Zapata public.
Zapata has gathered a few other high profile customers, too. BMW uses its tech to automate manufacturing plant schedules, for instance, the startup says. It generated $5.2 million in revenue in 2022 and was on track to outpace its 2023 revenue, it's public financials say, though they also said it lost $23.4 million from operations in 2022, and was narrowing those losses only a bit in 2023.
Other ways to do GenAI cheaper
There are a growing number of approaches to making GenAI less expensive and more energy efficient such as Small Large Language Models. Small LLMs use the same transformer approach, but use fewer parameters or dials that tell the model how you want it to perform. Small are less capable than their bigger counterparts yet work well for more targeted use cases.
But Zapata believes that, for many company uses, it's not just the size of the model that's the problem. It is the transformer tech itself.
The quantum physics world that Zapata's scientists belong to has been solving difficult computational problems that require a computer to generalize long before OpenAI's generative AI existed, Savoie says.
Such work has been generating outcomes ranging from new drug molecules to collider research using classic computer hardware. They will be able to use quantum computers, too, one day, if they ever prove reliable enough, Savoie says.
"We know how to do this math better," he said. It simply didn't occur to the quantum physics crowd to use their tech on language and art earlier because "the quantum physics crowd didn't interact with" computer scientists working on the transformer-type AI technology, Savoie added.
Zapata's founders, however, realized in 2018 that their tech could be used to generalize in other ways, such as which words to chose to answer to a question. Since founding that year, Zapata has 18 patents and patents pending on its AI tech, it says.
Others are working on Quantum for GenAI
Zapata isn't targeting consumer users, though. It competes more directly with AI companies like Anthropic and Cohere, where a company uses its own data to train their models. OpenAI also offers an enterprise version which can be trained using corporate data.
"That honeymoon period of kicking the tires I think is on its way out and now it's about production," Savoie says. "That's when they come to us and it's like, 'oh crap, our OpenAI model is going to cost a gazillion dollars. It's going to cost more than it's actually worth.'"
Zapata is not the only company working on using quantum physics for generative AI. Computer giants working on quantum computers are also dabbling such as Fujitsu, Google, IBM, Meta, Microsoft.
While Zapata does claim that the model training and operation is less expensive, it's also not exactly a cheap, DYI option. Companies can use Zapata's tech as a cloud service that runs on AWS or other clouds, or as software that runs in their own data center, but they also typically need to hire Zapata consultant services to set them up. "We bundle them. That's how our business model works. It's very much like Palantir," Savoie said.
Zapata previously raised $64.7 million in venture capital from such backers as Comcast Ventures and climate-focused fund Prelude Ventures, the company said. The date of the merger hasn't been announced but the deal is expected to value the new public company at around $200 million, Bloomberg previously reported.
Whether Zapata is ultimately successful in its business model, time will tell. But the idea that there are more efficient ways to do generative AI than the current more-is-better state of the art will surely be an ongoing theme. And quantum tech could be among the alternatives.
