The Six-Month Chip

A custom AI chip can reach production two years after its designers forecast the workload it needs to run.

Two years ago, the dominant model architecture looked different. The balance between training and inference looked different. Context windows, quantization, and agent workloads looked different. Designing a bespoke accelerator is a wager that today’s bottleneck will still matter after several generations of models have come and gone.

Meta’s answer is to compress the hardware cycle until it starts to resemble a software roadmap. The company says it can now ship a new generation of its Meta Training and Inference Accelerator roughly every six months, with four MTIA generations planned across two years.

The headline is custom silicon. The more interesting story is how to keep custom silicon from becoming custom obsolescence.

The workload moves before the chip arrives

Meta’s MTIA roadmap begins with recommendation systems and shifts toward generative-AI inference as it progresses.

MTIA 300 started around ranking and recommendation workloads. MTIA 400 broadened toward generative AI. MTIA 450 and 500 are designed first for generative-AI inference. Meta says it has already deployed hundreds of thousands of MTIA chips and tested the stack with Llama.

The sequence is a record of workload drift. The chip program survives because its parts are reusable: modular chiplets, common software, chassis, racks, and network infrastructure. A generation can change compute, memory, or communication without redesigning the data center from zero.

That is what six-month cadence really means. Not a fresh chip factory twice a year, but a platform that limits how much must change at once.

Memory is becoming the visible constraint

Inference has two broad phases. Prefill processes the input context. Decode emits tokens one after another, repeatedly moving model weights and attention state through the system.

More arithmetic helps, but decode frequently waits on memory movement. Long contexts grow the key-value cache. Larger models need more capacity. Parallel agents turn one user request into many simultaneous sequences. Interconnect matters when work spans chips.

Meta’s roadmap follows those pressures.

• MTIA 400 has four times the FP8 compute and 51 percent more high-bandwidth memory throughput than MTIA 300, according to Meta.
• MTIA 450 doubles memory bandwidth again versus 400 to accelerate decode, adds 75 percent more MX4 compute, and includes hardware aimed at attention and feed-forward bottlenecks.
• MTIA 500 adds another 50 percent of memory bandwidth, up to 80 percent more memory capacity, and 43 percent more MX4 compute versus 450.

Across MTIA 300 to 500, Meta reports memory bandwidth rising 4.5 times and compute rising 25 times. The compute comparison changes precision—from MX8 to MX4—so it is not an apples-to-apples FLOPS chart. That footnote is the point: low precision is part of the architecture, not a formatting detail.

The network moved onto the package

Meta’s newest disclosure, published August 24, puts twelve custom 800-gigabit RDMA network interfaces across two MTIA 300 chiplets. Meta says that supplies 1.2 terabytes per second of total I/O without crossing PCIe, with dedicated communication engines handling collective operations.

On one 150-billion-parameter recommendation model across 40 accelerators, Meta reports communication 3.9 times faster than an “equivalent GPU cluster.” That is a workload-specific vendor comparison, not a general result. It still shows where the company spent silicon: not only on matrix multiplication, but on keeping a distributed system fed.

Google’s Ironwood TPU makes the same priority visible at another scale. Current TPU7x specifications list 192 GiB of high-bandwidth memory and 7.38 terabytes per second of memory bandwidth per chip, with pods scaling to 9,216 chips. Groq takes a different route, emphasizing on-chip SRAM and deterministic dataflow to reduce the cost of moving weights.

Three architectures, one constraint: useful inference depends on where the data is when the compute needs it.

Owning the chip is not automatically a moat

Custom silicon can remove a supplier’s margin and tune a fleet for a stable, high-volume workload. It can also lock a company into yesterday’s assumptions.

The owner absorbs design cost, compiler work, verification, manufacturing yield, packaging, deployment, and the risk that a model change undermines the bet. A specialized accelerator with poor utilization is an expensive monument. A fast chip without compiler coverage is inventory.

The durable advantage is the feedback loop: production workloads reveal a bottleneck; compiler and model teams expose it; hardware targets it; the data center can deploy the change; utilization data shapes the next generation. Meta’s six-month claim is valuable only because the same company controls enough of that loop to act on it.

GPUs retain their advantage precisely where workloads are uncertain. Generality is not inefficiency when the alternative is an obsolete fixed function.

How to buy inference without buying a slide

Peak FLOPS cannot price a token. Ask vendors for measurements under your workload:

• Prefill and decode reported separately.
• Median and p95 inter-token latency.
• Throughput at target batch size and concurrency.
• Memory capacity, bandwidth, and key-value cache policy.
• Power measured at the wall.
• Quantization quality on your model.
• Compiler and kernel coverage.
• Cost per successful production request.

Then test portability. Architecture-specific savings can vanish if the next model needs an unsupported operator, more memory, a different numerical format, or a new serving pattern before the hardware is amortized.

The long game is not GPU versus custom chip. It is whether the hardware roadmap can keep learning at the speed of the workload.

Meta is trying to make a chip every six months. The real product is the system that makes the sixth chip less of a guess than the first.

---

All roadmap dates and performance figures above are vendor-reported. Planned 2027 deployments are not completed deployments, different numerical precisions are not interchangeable, and bandwidth is only one part of inference performance.