The close relationship between OpenAI and Nvidia is fluctuating.
Eight sources familiar with the matter told Reuters that artificial intelligence firm OpenAI is unhappy with some of Nvidia's latest AI chips and has been seeking alternatives since last year, potentially complicating the relationship between the two leading AI companies. This shift in strategy by the company developing ChatGPT is characterized by an increased focus on chips used to implement specific elements of AI inference—the process by which an AI model, such as the one powering ChatGPT, responds to user queries and requests. Nvidia remains dominant in large-scale model training chips, while inference has become a new frontier in the competition. OpenAI's decision, and that of others, to seek alternatives in the inference chip market represents an important test of Nvidia's dominance in artificial intelligence, and comes at a time when the two companies are in investment talks. In September, Nvidia announced its intention to invest up to $100 billion in OpenAI, in a deal that would give it a stake in the startup and provide the AI company with the liquidity needed to purchase advanced chips. The deal was expected to close within weeks, according to Reuters. But negotiations dragged on for months. During that time, OpenAI struck deals with AMD and other companies to acquire graphics processing units (GPUs) designed to compete with Nvidia's chips. However, its changing product landscape also altered the type of computing resources it required, which hampered talks with Nvidia, according to a person familiar with the matter. Nvidia CEO Jensen Huang on Saturday denied any tension with OpenAI, calling the notion "nonsense" and confirming that Nvidia plans a major investment in OpenAI. Nvidia said in a statement: "Customers continue to choose Nvidia for guidance because we deliver best performance and lowest total cost of ownership across a broad range." An OpenAI spokesperson said in a separate statement that the company relies on Nvidia to operate the vast majority of its inference fleet, and that Nvidia offers the best cost-performance ratio for inference operations. Following the publication of the Reuters report, OpenAI CEO Sam Altman wrote in a post on the X platform that Nvidia makes "the best AI chips in the world" and that OpenAI hopes to remain "a very large customer for a very long time." Seven sources have stated that OpenAI is dissatisfied with the speed at which Nvidia's hardware can generate responses for ChatGPT users in certain types of tasks, such as software development and AI communication with other software. The company needs new hardware that could eventually provide approximately 10% of OpenAI's future computing needs for inference, one source told Reuters. Two sources said OpenAI had discussed collaborating with startups, including Cerebras and Groq, to provide chips that would speed up inference. But one of the sources told Reuters that Nvidia had struck a $20 billion licensing deal with Groq, effectively halting OpenAI's negotiations. Chip industry executives said Nvidia's decision to poach key talent from Groq appeared to be an attempt to bolster its technology portfolio and better compete in the rapidly evolving artificial intelligence industry. Nvidia stated that Groq's intellectual property complements Nvidia's product portfolio significantly. Nvidia alternatives
Nvidia's graphics processing chips are well-suited for handling the massive amounts of data needed to train large AI models like ChatGPT, which have contributed to the explosive growth of AI globally to date. However, advancements in AI are increasingly focused on using trained models for reasoning and inference, potentially representing a new and broader phase in AI development, and inspiring OpenAI's efforts. OpenAI's research into alternatives to graphics processing units (GPUs) has focused since last year on companies that manufacture chips with large amounts of memory integrated into the same silicon chip as the rest of the chip, called SRAM. This high concentration of expensive SRAM on each chip can provide speed advantages for chatbots and other AI systems as they process requests from millions of users. Reasoning processes require more memory than training because the chip needs to spend a relatively longer time fetching data from memory compared to performing calculations. Nvidia and AMD's GPU technology relies on external memory, which increases processing time and slows down user interaction with chatbots. One source said the problem was particularly pronounced within OpenAI's Codex product, a code-building tool the company heavily markets. Another source indicated that OpenAI employees attributed some of Codex's weaknesses to Nvidia's graphics processing unit (GPU) hardware. In a call with reporters on January 30, Altman said that customers using OpenAI's programming models would give "a high priority to speed in programming work." Altman explained that one way OpenAI will meet this demand is through its recent deal with Cerebras, adding that speed is less important for ordinary ChatGBT users. Competing products such as Anthropic's Cloud and Google's Gemini benefit from deployments that rely more heavily on Google's internally manufactured chips, known as Tensor Processing Units (TPUs), which are designed for the computational operations required for inference and can offer performance advantages compared to general AI chips such as Nvidia's GPUs.
0 Comments