The history and future of AI at Google, with Sundar Pichai

Sundar Pichai reflects on Google's AI productization and the emergence of ChatGPT

Google developed Transformers to address specific product needs, such as enhancing translation and scaling speech recognition, leveraging TPUs. They immediately integrated this research into Google Search, which led to substantial improvements in search quality through models like Bert and Mom, demonstrating early and effective application of the technology.

Internally, Google also developed Lambda, an early generative AI project akin to ChatGPT, which an engineer at the company believed was sentient. A constrained version of this technology was later introduced publicly as AI Test Kitchen at Google I/O 2022, showcasing their parallel progress in generative AI.

The unconstrained internal version of Lambda was deemed "toxic," and Google's high quality standards, particularly its search quality bias, delayed a public release compared to OpenAI's ChatGPT. The initial ChatGPT launch itself was a relatively quiet event, and OpenAI had secured a deal with Microsoft months prior.

Sundar Pichai views such competitive developments as typical in the consumer internet space, citing historical examples like YouTube emerging after Google Video Search, or Instagram after Facebook. He emphasizes that new products constantly emerge from various teams, and companies must be prepared for these surprises.

I don't think people like wake up in a garage and ship a better iPhone. Like that's not going to happen. But that's not how consumer Internet is.

Google prioritizes speed and low latency as a core differentiator across its products, including its AI offerings.

Google has historically differentiated its products through speed and low latency, a strategy evident in original Google Search, Gmail, and Chrome. This long-standing focus continues today with its AI services, such as Gemini running on TPUs, which are designed for rapid performance.

Internally, Google views latency as a critical distinguishing feature of a great product, often reflecting well-engineered technical foundations. This requires a careful balance between constantly adding new capabilities and maintaining high speed.

For instance, Google Search teams operate with specific "latency budgets" in milliseconds. If a sub-team develops a feature that reduces latency by 3 milliseconds, they earn 1.5 milliseconds of credit for their budget, which translates directly to a faster experience for the user. This rigorous approach has led to a 30% improvement in search latency over the past five years, even as functionality has increased.

In the AI domain, Google applies this principle to Gemini by developing "Flash models." These models aim to achieve 90% of the capability of the larger Pro models but operate significantly faster and are more efficient to serve, reflecting a strategic focus on the Pareto frontier of capability versus speed.

To give an example, like search, I was speaking with the teams they now have for sub teams, like latency budgets. In the milliseconds, you get 50% credit. So if you ship something which shaves off 3 milliseconds, you earn 1.5 milliseconds for your latency budget and 1.5 milliseconds gets passed on to the user.

The Future of Search: Evolving into Agentic Systems

Many now discuss chat and agentic flows as the future interface, moving beyond simple queries. Instead of typing a search query, a personal agent might go and plan an entire trip. Google's search is already incorporating Gemini and AI results, allowing people to perform deep research queries that diverge from traditional short prompts.

Search has consistently evolved with technological shifts, such as adapting quickly to mobile when users needed to find information on the go. The future will see information-seeking queries transform into agent-taken search, where users complete complex tasks and manage multiple threads simultaneously. The product frontier must continuously absorb new capabilities to meet shifting user expectations.

The current rapid pace of AI development creates an "expansionary moment" rather than a zero-sum game. The value derived from these new capabilities is growing exponentially. Companies like Google are embracing this by evolving both traditional search and developing advanced AI models like Gemini, viewing the overlap and divergence as an opportunity to stay at the cutting edge.

I feel like in search with every shift you're able to do more with it and we have to absorb those new capabilities and keep evolving the product frontier.

Google's Long-Term AI Investments and Investor Sentiment

A year ago, investor sentiment towards Google was largely negative, with many believing its core search business was vulnerable. Google's stock was trading around $150 a share, reflecting a misunderstanding of its deep, foundational investments in artificial intelligence.

Sundar Pichai explains that Google was well-prepared for the AI shift, having operated as an

AI first

company since 2016. This included developing seven versions of TPUs and building AI data centers, demonstrating a long-term, intentional commitment to the technology.

I felt like the company was built for that moment. The vertical thing. It's not an accident or something. It was a very intentful.

Identifying Early Moments of General AI Progress

Sundar Pichai describes his first 'AGI moment' as occurring in 2012, when Jeff Dean demonstrated an early version of Google Brain. This was a significant event where neural networks successfully recognized a cat, marking a tangible step forward in artificial intelligence.

Other notable instances included witnessing cars drive autonomously at the DARPA challenge around 2014 and seeing Demis's early models exhibit what could be described as 'imagination.' These events provided a visceral sense of AI's accelerating progress over the years.

More recently, a 'field AGI' moment arises when a user gives an AI a complex coding task and observes it complete the work autonomously, without needing to open the IDE or intervene. This demonstrates a powerful capability akin to an agent manager handling intricate projects independently.

The rapid 'slope of the curve' in AI development is particularly striking, with improvements happening across numerous paradigms. This consistent and broad progress strongly suggests that significant advancements in AI are still ahead.

My first feeling, the AGI moment was 2012 when Jeff Dean demoed the earliest version of Google Brain. This is when the neural networks recognized a cat.

Leaders Stay Connected to Product Experience

Leaders at tech companies must stay intimately connected to their products' user experience, as solely relying on reports, slide decks, and spreadsheets can be insufficient for understanding abstract tech products. This direct connection helps identify hidden frustrations and improve overall product quality.

One approach is to engage in regular product use, as exemplified by Tony Hsu working as a DoorDasher. The host's company also dedicates weekly "Just walk the Store" segments during All Hands meetings, where they collectively click through their dashboard to discover usability issues.

Sundar Pichai emphasizes "dogfooding," actively using internal versions of products. He blocks dedicated time for intense usage, such as spending 30 minutes talking to Gemini Live on a single topic, which reveals both useful and frustrating aspects of the product.

He also leverages external platforms like X for raw, unfiltered user feedback, noting its effectiveness in flagging issues like a recent Google Calendar fix. Internally, he queries systems like Anti Gravity after launches, specifically asking for the top five best and worst things people are saying about a new feature to gain comprehensive insight.

You cannot just manage through reports from teams and slide decks and spreadsheets.

AI's Economic Impact and Market Growth

The speaker notes personal productivity gains from AI agents, which simplify tasks and reduce time previously spent on manual efforts. This individual experience reflects the broader efficiency enhancements expected across various industries.

Unlike prior technology cycles such as the Internet, mobile, or SaaS, which took a while to manifest in GDP numbers, AI's economic contribution is already apparent. Data center build-outs specifically are driving a portion of current GDP growth.

Despite past skepticism regarding the high capital expenditure required for AI, the speaker is now confident in its potential. Demand for AI services is currently high and outstripping supply, indicating a massive and growing market.

I actually don't have any doubt that this is a massive market and outcome.

The software engineering market has significant untapped growth potential.

One speaker suggests the market for software engineering and coding is much larger than commonly perceived. They argue that the traditional metric of comparing token budgets to engineer salaries is flawed, as increased supply could dramatically expand the market by a factor of ten.

Sundar Pichai acknowledges that economic metrics like GDP don't always capture the full impact of technological shifts, similar to how the internet's true influence isn't fully reflected in past GDP growth.

However, Pichai also highlights natural constraints on growth and societal diffusion. He points to limitations in compute infrastructure build-out and the responsible pace at which technologies, like Waymo's self-driving cars, can be integrated into society, even when proven safer than human drivers.

Despite these dampening factors, Pichai believes that even a modest increase in growth, such as a half percentage point in the large US economy, represents a substantial contribution from technological advancements.

I think the market for software engineering and coding is dramatically bigger than anybody thinks.

Supply Constraints Impact AI Compute Scaling and Market Dynamics

Despite massive capital expenditure commitments, such as Google's approximately $180 billion in capex, the ability to scale AI compute is significantly hindered by real-world supply constraints. Even if companies like Google wanted to spend $400 billion, components like memory and sufficient power infrastructure are currently unavailable, particularly for 2026 and 2027.

The primary bottlenecks include fundamental wafer capacity, the speed of permitting and regulatory approvals for new data centers, and the availability of critical components like memory. While power and energy supply issues are seen as more solvable, the pace of construction, especially in the US, is a concern when compared to regions like China.

These constraints create a 'musical chairs game' for compute resources, potentially capping how far ahead any single company can pull relative to others. This environment forces companies to make adjustments based on error bands and extraneous factors, driving a strong capitalist incentive for innovation to improve efficiency, such as making models 30 times more efficient. However, these incentives cannot fully overcome short-term physical supply limitations.

The ongoing supply limitations, particularly for memory, may lead to market divergence in the mid-term (2026-2027), where some companies will simply not be able to acquire the necessary resources. This situation highlights that despite technological advancements, the physical world's capacity to produce essential hardware remains a critical bottleneck.

It's kind of similar with memory where ultimately some people have to not get the memory they want.

Artificial Intelligence Poses Significant Security Constraints on Software

Artificial intelligence models are expected to significantly challenge the security of nearly all existing software. The powerful capabilities of these models could potentially uncover numerous vulnerabilities, leading to an increase in what are known as zero-day exploits.

A notable indicator of this trend is the reported decline in the black market price for zero-day exploits. This drop is attributed to an increased supply, driven by AI's enhanced ability to discover these critical vulnerabilities, which represents a substantial shift in the cybersecurity landscape.

While these emerging security challenges present new constraints on software systems, they may also serve as a catalyst for innovation and efficiency. The necessity to address these AI-driven threats could stimulate crucial discussions and foster greater coordination within the cybersecurity community, which is currently lacking.

These models are definitely really going to break pretty much all software out there maybe already, we don't know.

Sundar Pichai on Google's Long-Term Investments in Emerging Technologies

Google maintains an impressive portfolio of long-term projects and strategic investments, including stakes in SpaceX and Anthropic, alongside internal ventures like Waymo and Quantum computing. Sundar Pichai explains that these initiatives, which might initially seem audacious, are pursued with a 20-year outlook, drawing parallels to Waymo's early days. The company even considers highly ambitious concepts like data centers in space, using such constraints to foster innovative solutions.

Quantum computing is a core commitment for Google, driven by the belief that it can fundamentally enhance the simulation of nature, which is inherently quantum. While common discussions focus on molecular modeling and cryptography, Pichai suggests that the full spectrum of Quantum's applications, much like mobile phones enabling Uber, will likely be discovered creatively once the technology matures and becomes widely accessible. He anticipates Quantum will have an advantage in simulating reality, weather, and complex natural processes.

Google is also re-engaging with robotics, recognizing that advancements in AI, particularly models like Gemini Robotics, provide the missing ingredient that was absent years ago. The company is actively partnering with external firms like Boston Dynamics and Agile. Furthermore, the Wing drone delivery service is scaling up significantly, aiming to provide service to 40 million Americans soon. Another key initiative is Isomorphic Labs, which focuses on targeted models to enhance drug discovery processes, aiming for much higher success rates beyond just molecular design.

I'm confident Quantum will have many, many, many applications if you can actually make it work.

How Google Allocates Capital Across Diverse Projects

Google faces a unique challenge in capital allocation, needing to compare and fund highly heterogeneous projects. Unlike traditional businesses where choices might be between similar ventures, Google evaluates options ranging from improving YouTube's recommender algorithms for immediate monetization to scaling Waymo for faster market entry, or investing in nascent AI approaches with five-year payoff horizons. These projects have vastly different natures and payoff curve shapes, making direct comparison difficult.

Sundar Pichai explains that Google's approach involves making early bets on deep technology. At these initial stages, funding amounts can be smaller, and evaluation focuses on underlying technical progress rather than immediate returns. For instance, in quantum computing, they assess whether a team is meeting goals like achieving stable logical qubit thresholds, ensuring progress in a deep and fundamental way.

A core tenet of Google's strategy is assessing the long-term value of initiatives. This involves intuitively considering the 'option value' and the total addressable market (TAM) five to ten years in the future, often assuming significant growth. This long-term perspective allows Google to commit to projects like TPU development or Waymo, even increasing investment when external sentiment turns pessimistic, as was the case with Waymo a few years ago.

I always view it as you have to assess the long term value of these things. So it's almost like in some intuitive way you're thinking about the option value and the TAM of something five to ten years down the line and you assume a crazy growth and think through whether those decisions make sense.

Google's decision-making process for high-risk, long-term projects like Waymo

Google frequently cuts projects that aren't working, but Waymo, despite a long development path from demo to commercial service, was sustained. The decision to keep Waymo and discontinue projects like Loon was based on a quantified evaluation of the underlying technology, specifically the Waymo driver's progress in safety and reliability.

Evaluating progress at a deeper technological level is crucial, even through phases where development may slow. Confidence in the team's ability to overcome challenges and breakthroughs, such as the shift from hand-mapped heuristics to end-to-end deep learning in Waymo, were critical for advancing the technology.

For complex, integrated systems like Waymo, which can be viewed as a robot, there are hidden aspects and craftsmanship that matter for long-term development. The host notes that while robotics might move faster now, Waymo's long development fostered a unique level of system integration.

Sundar Pichai emphasizes that to significantly advance the curve, especially in areas with safety and regulatory concerns, having firsthand experience with the product feedback cycle is vital. This often necessitates having first-party hardware, a lesson learned from projects like Waymo and the development of TPUs for AI.

My lesson from Waymo and on the AI side with TPUs, et cetera, I think to really push the curve well, particularly in areas where you have safety, regulatory everything you want the firsthand experience of the product feedback cycle.

Assessing Google's Historical Capital Allocation Strategy

Google has historically maintained a strong net cash position, leading to questions about whether the company has been underlevered. Given its abundance of ideas, the durable growth of its core business, and a clear understanding of its operations, there's an argument that Google could have been more aggressive in deploying capital towards new initiatives, share buybacks, or minority investments.

Sundar Pichai emphasizes Google's commitment to being a good steward of capital, citing investments in companies like Stripe, SpaceX, and Anthropic. He notes that the current shift in AI presents even more opportunities for effective capital deployment, which Google is actively pursuing.

Regarding specific projects, Pichai mentioned that he would have liked to invest more capital into Waymo earlier. However, the project had not reached the necessary level of maturity, particularly from a safety standpoint, to justify earlier, larger investments. This indicates that the pace of capital deployment is often tied to a project's readiness and conviction, rather than just capital availability.

I would have been glad to invest more capital in Waymo earlier, but we weren't at the level of maturity needed to do that.

Google's Approach to Allocating Scarce Compute Resources

Historically, R&D expenses at tech companies were dominated by employee headcount, with tech costs largely secondary unless a project was extremely computationally intensive. Google, however, is now operating in a world where advanced compute resources like TPUs and GPUs represent a significant and increasingly constrained capital allocation challenge.

Internally, Google manages its compute budget meticulously. Sundar Pichai dedicates a dedicated hour each week to review compute unit usage at a granular level, understanding which projects and teams are consuming resources. This intense focus ensures that Google's valuable compute is directed towards the most critical initiatives.

The scarcity of compute creates a unique challenge for Google Cloud Platform (GCP), where Google allocates these same resources to external customers. Google addresses this by significant forward planning, anticipating both internal and external needs.

Any compute commitments made to GCP customers through long-term contracts are considered sacrosanct. This contractual obligation necessitates robust planning to balance the demands of internal projects with the promised resources for cloud clients, even in a constrained environment.

Anything we commit to a customer is sacrosanct. Right. So these are contractual commitments.

AI simplifies navigation and interaction with extensive cloud platforms like Google Cloud Platform.

Large cloud platforms, such as Google Cloud Platform (GCP), offer a vast array of functionalities, which paradoxically can make them challenging for users to navigate. Users often find it difficult to locate specific services, manage permissions, or set up new projects within such broad ecosystems.

AI is emerging as a crucial solution to this complexity, acting as an orchestration layer that enables programmatic interaction with cloud services. By processing extensive API documentation, AI can translate user requests into effective actions, significantly streamlining access to features and making complex systems more manageable.

This AI-driven approach is especially beneficial for products with a large operational surface area. The same navigational challenges faced by GCP are also experienced by platforms like Stripe, which continually add more functionalities. AI helps users efficiently thread through expansive documentation and service offerings.

Ultimately, AI's role as an intelligent orchestration layer simplifies how users interact with vast amounts of data and services. This not only enhances user experience but also negates the need for extensive, traditional integration projects often associated with connecting disparate systems, such as large ERP implementations.

AI being this orchestration layer in a way that makes sense for the end user, I think has been delightful to see.

Stateful AI for Consumers and Persistent Applications

Mainstream AI applications currently lack the ability to maintain state or persistence over time, preventing them from handling long-running, customized tasks. The emerging concept of stateful AI for consumers aims to address this by allowing applications to remember user preferences and past interactions, enabling more personalized and continuous experiences.

This shift opens up possibilities for what is termed an "agentic future," where AI can perform complex, persistent tasks reliably and securely. Examples include an AI that consistently rounds up daily news tailored to individual interests. Implementing such systems requires careful consideration of identity, access, and security to ensure user data protection.

Companies like Dreamer, which was acquired by Meta, have already explored this space by enabling users to design custom software with built-in persistence. This demonstrates the potential for consumer interfaces to evolve, incorporating full coding models with the necessary tools and secure cloud-based persistence. The goal is to make these advanced capabilities accessible to a much wider audience beyond just developers.

While only a small fraction of the world currently experiences this future, where they can build custom, persistent AI tools for themselves, the aspiration is to bring this functionality to mass adoption. This represents an exciting new frontier for consumer technology, promising more dynamic and personalized AI interactions.

bringing that to mass adoption is a very exciting frontier, I think.

Google Docs Search Challenges and Upcoming AI Improvements

Users often find searching within Google Docs significantly harder than searching Gmail because common keywords, such as "2026 budget" in a presentation, are not unique enough to pinpoint specific documents. This contrasts with email, where keywords are typically more distinct, making it easier to locate exact messages.

Sundar Pichai acknowledges this specific search problem resonates with user experiences and confirms that Google is actively working on addressing these difficulties. He indicates that the company recognizes the need for better document retrieval within its productivity suite.

Looking ahead, Pichai highlights that AI integration will bring substantial improvements to Google Docs and related services. These advancements will focus on enhancing search by leveraging context and caching mechanisms, with "sharp improvements" expected in the coming months.

I think, look, the AI integration into these services, including Google Docs, I think you will see sharp improvements in the coming months ahead.

Google's Workflow Shifts with New Technologies

Many companies are dramatically overhauling their workflows for product development and engineering practices, even re-evaluating design team compositions. Google is also navigating this transformation, with certain groups already making profound shifts in their operational approaches.

Google's CEO, Sundar Pichai, describes the internal adoption of new workflows as concentric circles, where diffusion to more groups is a significant task. Initially, some new tools were 'semi broken,' making early widespread adoption challenging. However, the curve is shifting dramatically now, indicating improved reliability and readiness.

Specific groups within Google, such as GDM and some SWE teams, have notably changed their workflows. They utilize an internal product called "Jet Ski" (known externally as "anti gravity"), which facilitates an "agent manager world" way of working. This new workflow was recently rolled out to Google's search team.

Diffusing new technologies and managing organizational change is a particularly challenging aspect for a large company like Google. While smaller companies might switch over quickly, a large organization faces a more complex process in adopting these significant workflow changes.

Change management is a hard aspect of this technology diffusing, which may be easier for a small company.

Companies Encounter Significant Hurdles in Widespread AI Adoption and Diffusion

Despite the abstract capabilities of AI, companies face significant challenges in diffusing it widely across their operations. A primary hurdle is the time engineers need to master prompting AI, which involves both general techniques and specific knowledge of company tools, like Stripe's internal systems. This learning curve impedes efficient AI integration.

Collaborating on AI-generated code also presents difficulties. The rapid changes and potential "blast radius" of such code mean it often requires multiple rewrites before deployment. This high turnover makes it harder for teams to work together effectively compared to traditional, slower code development.

Outside of engineering, enabling AI agents to answer common business questions, such as "what's the status of this deal," requires extensive work on data access. Companies need to rethink and rewrite their entire permissions infrastructure to ensure AI agents can securely and appropriately access internal data.

Google's internal teams, including Gemini enterprise, are actively tackling these very diffusion problems by using AI internally, identifying friction points, and developing solutions. They prioritize robust security and identity access controls, acknowledging these add a "fixed cost" but will lead to more reliable AI deployment across the company.

I definitely expect in some of these areas 27 to be important inflection point for certain things. Even the people doing it, that is the workflow through which they would produce it. And maybe for a while you would check it in the conventional way, but you kind of switch over crossover. But I expect 27 to be a big year in which some of those shifts happen pretty profoundly.

Sundar Pichai on Small Google Initiatives

Google's CEO, Sundar Pichai, highlighted the company's approach to fostering innovation by starting small, even for ambitious projects. This strategy mirrors the origin of significant advancements like the Transformer, which began as a minor initiative within Google.

Pichai offered two examples of small-scale projects he finds exciting. One is the concept of "data centers in space," which began with a small team and limited budget to achieve initial milestones. This demonstrates how even grand ideas are given room to incubate on a smaller scale.

Another area of excitement for Pichai involves an improvement in "post training" within machine learning, developed by a single individual. He described this as a significant leap forward, underscoring the impact that focused, individual contributions can have on Google's technological progress.

These examples illustrate Google's belief that major breakthroughs often originate from small, experimental teams and individual efforts rather than immediately committing vast resources. This allows for agility and testing of concepts before wider adoption.

It's important to start small even if it's a big idea.