Pi’s Inflection-2: Rivaling Google, Meta, and Gaining on GPT-4

Inflection AI’s latest breakthrough, the Pi chatbot, reportedly surpasses Google and Meta’s AI models, rapidly closing in on OpenAI’s renowned GPT-4.

Inflection AI’s new AI model, Inflection-2, has outshined Google’s PaLM Large 2 and Meta’s LLaMA 2 in several benchmarks, positioning itself just behind OpenAI’s GPT-4 in terms of performance. CEO Mustafa Suleyman expressed excitement about future AI advancements, calling the current progress just the start of an incredible journey.

Inflection-2 will soon enhance Pi, the company’s conversational chatbot released earlier this year. The integration involves aligning the model with Pi’s unique communication style and ensuring it remains up-to-date and accurate, reducing misinformation.

Suleyman highlighted Pi’s ability to handle sensitive topics like race, gender, and politics with factual accuracy and real-time information. An update to include Inflection-2 is imminent, promising to further boost Pi’s capabilities, though a specific date hasn’t been announced.

While user statistics for Pi remain undisclosed, its popularity and retention are reportedly high. This development comes amid significant events at OpenAI, including the temporary dismissal and subsequent reinstatement of CEO Sam Altman. Inflection AI’s latest release positions them prominently in the competitive landscape of AI language models.

At Inflection, CEO Suleyman clarified that the release timing of their new model, Inflection-2, wasn’t expedited despite prior hints at a year-end launch. In fact, the release was slightly delayed, coming a week later than planned after training completion last week.

To develop Inflection-2, the company utilized 5,000 of Nvidia’s H100 GPUs, a step up from the A100s used for its previous model. This upgrade resulted in a model that was not only faster but also more cost-efficient to train, yet still handling a staggering 10^25 FLOPs. Inflection’s collaboration with Microsoft, Nvidia, and CoreWeave was crucial in managing its extensive compute cluster.

In its testing phase, Inflection-2 was benchmarked against the MMLU, a set of advanced tasks spanning 57 subjects, ranging from general world knowledge to specific problem-solving and ethical scenarios, typically encountered at high school and professional levels.

Suleyman highlighted that Inflection-2 surpassed several major AI models, including the 70 billion parameter LLaMA 2, Elon Musk’s xAI startup’s Grok-1, Google’s PaLM 2 Large, and Anthropic’s Claude 2, only falling behind GPT-4. In scientific answering benchmarks, Inflection-2 outdid LLaMA 2 and PaLM 2 in all but two out of seven categories. Additionally, it led in two out of three Q&A task benchmarks, with PaLM 2 Large edging it out in one.

Notably, Inflection-2 also showed commendable performance in math and coding benchmarks, areas that were not its primary focus. While it couldn’t quite match GPT-4 in the two benchmarks shared by OpenAI, its overall performance was respectable, demonstrating its robust capabilities across a variety of tasks.

Suleyman emphasized that while AI benchmarks might seem trivial to those outside the field, they are critical in differentiating between a basic prototype and a high-quality, reliable model. He proudly positioned Inflection-2 as a top contender in its class, nearly matching OpenAI’s GPT-4. With plans to scale up its next model by 10 times in six months, and another 10-fold increase following that, Inflection is ambitiously expanding.

Amidst the turmoil surrounding OpenAI’s board, Suleyman called for empathy, highlighting the good intentions of many involved, including Ilya Sutskever, with whom he shares a history and respect from their time at DeepMind.

Inflection, under Suleyman’s leadership, has been a vocal advocate for AI safety, aligning with initiatives like President Biden’s voluntary order. Staying clear of political campaigning and election-related queries with their AI chatbot Pi, Suleyman also authored “The Coming Wave,” addressing AI risks.

While acknowledging OpenAI’s recent challenges as minor setbacks in the grand scheme, Suleyman remains a driven entrepreneur. Recognizing the competitive nature of the industry, he sees Inflection’s growth as both a business venture and a part of the broader, innovative landscape of Silicon Valley.