At Google Cloud Next last month, the company announced not one AI chip, but two. The TPU 8t for training, and the TPU 8i for inference. Different silicon, different architectures, different workloads.

That split is the next step in the maturity of AI deployment and the realization that AI training and AI inferencing are inherently different problems with different architectural needs.

The 80 Percent Cost Problem

Google’s TPU 8i delivers an 80 percent performance-per-dollar improvement over the previous generation for inference. This is a fundamental architecture mismatch being corrected at the silicon level.

Why does inference need its own chip? Because training is about throughput; inference is about latency. The TPU 8i triples on-chip memory and cuts communication delays five-fold specifically to keep the compute cores from sitting idle waiting for data. You can’t optimize for both workloads in the same architecture without compromising one.

Here’s what that means for your budget: model training is expensive but intermittent. Inference runs 24/7 serving every customer interaction. An 80 percent cost reduction on your largest ongoing expense changes which workloads become economically viable to automate. If inference costs half as much, you can profitably serve twice as many use cases.

And if Google’s TPU 8i delivers 80 percent savings while AWS and Azure don’t have equivalent specialized chips yet, infrastructure cost becomes a first-order reason to choose your cloud platform rather than an afterthought.

Your Data Architecture Just Became the Bottleneck

Google also launched their “Agentic Data Cloud” — reframing the enterprise data platform from “static repository” to “dynamic reasoning engine.” That’s an admission that legacy data warehouses can’t keep up with how agents actually work.

Agents don’t run batch queries. They execute hundreds of database calls per task, traverse knowledge graphs, hit vector stores, and persist state — all in the same request path. If your data layer moves at batch-job speed while your inference layer expects millisecond latency, you just created a bottleneck that kills your agent’s responsiveness.

Google’s Cross-Cloud Lakehouse lets you query data across AWS, Azure, and Google Cloud without moving it. That’s not convenience — that’s acknowledging data gravity is real and your infrastructure has to work with it, not against it.

Storage Just Became Performance-Critical

When agents generate millions of tokens per session and persist state across long-running reasoning loops, object storage (built for infrequent access) and block storage (built for predictable database I/O) aren’t architected for the write-heavy, metadata-intensive patterns agents create.

Here’s where NetApp’s positioning matters: NetApp is the only storage vendor built into first-party cloud services across all three major hyperscalers — FSx for NetApp ONTAP on AWS, Azure NetApp Files on Azure, Google Cloud NetApp Volumes on Google Cloud. Not just marketplace offerings. Native infrastructure, billed directly by each hyperscaler.

When Google’s Cross-Cloud Lakehouse assumes your data doesn’t move clouds, you need storage that delivers consistent performance everywhere. Google can optimize Managed Lustre for TPU 8i, but only inside Google Cloud. AWS optimizes for Trainium, AWS-only. NetApp provides unified data services that work identically whether your infrastructure is in AWS, Azure, Google Cloud, or on-premises. Across every GPU, TPU and CPU in every cloud.

When your agent orchestration spans clouds — inference on Google’s TPU 8i, training on AWS Trainium, data lakes everywhere — NetApp is the only storage layer that works as a consistent primitive regardless of where compute runs. The hyperscalers can’t provide this; they’re incentivized to build cloud-specific solutions.

The Air-Gapped Implication

The same forces driving specialized inference silicon and cross-cloud data architectures apply even more to air-gapped, on-premises deployments for regulated industries.

You can’t send ITAR, HIPAA, or sovereign data to Google’s TPU 8i. But you still need low-latency inference, orchestration, and cross-system data access. The architectural pattern of inference-optimized compute, data platforms that move at agent speed, and storage that doesn’t bottleneck needs to work behind the firewall. For this NetApp has created two solutions for the enterprise: The AIPod for large-scale training/inferencing and the AIPod Mini for local inferencing at the edge. Both solutions run on NetApp’s proven ONTAP which provides hybrid cloud capability so that your data that sits on-prem can instantly connect and share its data to any cloud.

When Google splits their flagship AI chip in half and rebuilds their data platform for “agent scale,” they’re reacting to a fundamental mismatch between what infrastructure was built for and what production AI needs. The question isn’t “cloud or on-prem.” It’s whether your platform assumes inference is the workload, or treats it as an afterthought.

Whoever solves that for air-gapped deployments doesn’t win a niche — they define the infrastructure cycle for every industry where data can’t leave the building.