AMD Instinct MI350P: 144GB HBM3E in a PCIe GPU

There's a specific kind of enterprise AI problem that the hyperscaler conversation tends to erase: you have a real workload, a real budget, and a rack that was built in 2021 with 20–30 kilowatts of power headroom. Nobody's selling you an NVL 72 pod. Nobody's writing the press release about you. You are the market that gets treated like a rounding error.

AMD's new Instinct MI350P is, depending on how the software story resolves, the first serious attempt to treat that market like a market.

Patrick over at ServeTheHome has been tracking the MI350P across the show circuit — Dell Tech World, HPE Discover, Computex 2026 — and the throughline from his coverage is that this card is everywhere in OEM systems right now. That kind of floor presence usually signals that OEMs have done the math and like what they found. The question is whether buyers will too.

What you're actually getting

The MI350P ships with 144GB of HBM3E memory (high-bandwidth memory, the fast stacked stuff that makes AI inference hum) and delivers up to 4TB/s peak memory bandwidth — 3.6TB/s sustained under real workloads, per AMD's own spec sheet. It supports the new MX floating-point formats: MXFP8, MXFP6, and MXFP4. If those strings of letters mean nothing to you: they're lower-precision number formats that let you run bigger models on the same hardware by shrinking the numerical footprint of each weight. The practical upside is that models you couldn't fit before start fitting. The tradeoff is some accuracy loss, which for many inference workloads is totally acceptable.

The power envelope is flexible in a way that matters operationally. In a dedicated GPU server you can push it to 600W. Drop it into a standard dual-slot 2U server and it'll run at 450W — which is what makes it palatable to the racks that don't have liquid cooling and aren't getting any. It's a passively cooled, full-height full-length double-width card on PCIe Gen5 x16, which puts it in native alignment with current-generation server platforms like AMD's own EPYC 9005 Turin lineup.

One real limitation worth naming: the MI350P doesn't carry AMD's Infinity Fabric interconnect. That's the high-speed scale-up network on the OAM-format MI350X that enables direct, fast GPU-to-GPU communication within a system. Without it, multi-GPU coordination has to route through the PCIe Gen5 switch fabric. For inference workloads — especially single-model, memory-bandwidth-bound jobs — this isn't necessarily a dealbreaker. For training at scale, it's a ceiling. Patrick's framing is useful here: "you're going to want to have these on a pretty fast PCIe Gen5 switch, so therefore you don't have to go off the switch as much as possible to be able to go GPU-to-GPU communication."

The card also includes hardware video decode engines onboard — a quietly important detail as video analytics becomes a standard AI infrastructure use case. Decoding locally on the GPU without bouncing to the CPU saves both latency and compute headroom.

Not for the hyperscalers

Your problem is: you have a budget, you have a rack with finite power capacity, and someone on your team just asked why you're not running inference on something more capable than whatever you stood up eighteen months ago. The MI350P is built for that conversation. The CFO who controls your cage allocation doesn't need to approve a $10M rack-scale deployment — she needs to sign off on a 2U or 4U GPU server that fits in the space you already have.

Patrick puts the target squarely on enterprises "installing 10s or 100s or maybe 1,000 GPUs. They're not going to be installing 100,000 GPUs." That's most of the organizations actually running AI workloads in production today, and it's a segment that has largely been forced to choose between underpowered PCIe options and rack-scale systems that require purpose-built facilities.

The partitioning capability matters here too. Two MI350Ps in a 2U box can be carved into logical slices, effectively replacing a larger fleet of smaller cards. The density math changes what's achievable in constrained space.

Nvidia's weird gap — and why it's actually not weird at all

Here's the competitive picture that explains why OEMs are putting this card in everything: Nvidia doesn't have an HBM-based Blackwell PCIe GPU available as a public part. The B200 PCIe and B300 PCIe simply haven't materialized on the market. If you want Nvidia's Blackwell generation in a standard PCIe server today, you're buying the RTX Pro 6000 Blackwell Server Edition — a GDDR7-based card with 96GB of memory and, per Patrick's ServeTheHome coverage, around 1.6TB/s of memory bandwidth.

Run the slot math. Eight MI350Ps in a server gives you 1,152GB of total memory (144GB × 8). Eight RTX Pro 6000s gives you 768GB. That's roughly 400GB of total capacity difference across a fully populated system. And on bandwidth, the gap is larger still — AMD's HBM3E architecture delivers more than double the bandwidth per card compared to the GDDR7 alternative, per Patrick's figures.

For AI inference specifically — which tends to be memory-bandwidth-bound and memory-capacity-bound rather than raw-compute-bound — those two numbers are essentially the scorecard. Patrick frames it plainly: "I know for AI inference, especially when I go do my decode, I need memory bandwidth and run larger models, I need memory capacity."

Why doesn't Nvidia have an HBM Blackwell PCIe part? The cynical read: HBM supply is scarce and hyperscalers pay more for it. The generous read: Nvidia's product segmentation keeps rack-scale and PCIe-slot products from cannibalizing each other. Both are probably partially true. Either way, the gap is real, and AMD is standing in it.

ROCm is the asterisk you need to read

Here's what I'd actually tell you if you DM'd me asking whether your team should seriously evaluate this card: the hardware specs will get you excited and then your ML engineer is going to ask about ROCm, and that's where the conversation gets complicated.

ROCm is AMD's open-source software stack for GPU compute — the AMD equivalent of CUDA (Nvidia's proprietary compute platform that the entire AI ecosystem has been building on for 15+ years). The MI350P runs on ROCm. If your team has spent two years building CUDA-optimized pipelines, migrating is not a weekend project. Framework support has expanded, but the depth of the CUDA ecosystem — the tooling, the community, the Stack Overflow answers, the fine-tuned kernels — doesn't have a direct AMD equivalent yet. That's not a knock on AMD; it's just the reality of what a decade-plus of ecosystem lock-in looks like from the inside.

The reason this matters for the MI350P specifically: the hardware-level case for this card is legitimately strong, and the mid-market need it addresses is real. But software friction can neutralize hardware advantage faster than any spec sheet can offset it. Before you plan a procurement around those 144GB numbers, your team needs to do an honest audit of where your actual workloads sit on the CUDA-dependency spectrum. Some inference workloads are more portable than you'd think. Others are not.

The thing this actually is

The MI350P is AMD making a direct argument that the PCIe mid-market deserves Blackwell-generation hardware — and then backing it with specs that Nvidia currently can't match in the same slot. 144GB of HBM3E, flexible power scaling from 450W to 600W, and per-slot memory capacity that outpaces the available Nvidia alternative by 50%: that's not a consolation prize, it's a product decision.

Whether it becomes a genuine alternative to Nvidia lock-in in the mid-market or remains a compelling-on-paper spec sheet is going to be settled entirely by ROCm — by whether the software ecosystem matures fast enough to make the migration calculus work for organizations that aren't starting from scratch. The hardware has done its part. The rest isn't a trade show conversation.

---

Yuki Okonkwo covers AI and machine learning for Buzzrag. Source: ServeTheHome on YouTube, AMD Instinct MI350P overview. Patrick's coverage at ServeTheHome noted sponsorship disclosure for this video.