Mixture Of Experts

5 stories tagged Mixture Of Experts.

NVIDIA Nemotron Lightning Is Built for AI Grunt Work

NVIDIA's Nemotron 3.5 Lightning is a 30B MoE model built for the boring, essential work inside AI agents—tool calls, validation, and retrieval at speed.

Apple M3 chip with colorful neon glow border and text "You Need Mac For Local AI" on black background

How a 26B AI Model Now Runs in 2GB of RAM on a Mac

Yuki Okonkwo1 day ago

AI. Yuki Okonkwo1 day ago

How a 26B AI Model Now Runs in 2GB of RAM on a Mac

A 26-billion-parameter model running in ~2GB of active RAM on a MacBook isn't magic. It's two independent timelines finally crashing into each other.

A person wearing headphones points at technical architecture diagrams and research papers explaining the Kimi K3 model's…

Kimi K3 Architecture: KDA, MoE, and Attention Residuals

Dev Kapoor2 weeks ago

AI. Dev Kapoor2 weeks ago

Kimi K3 Architecture: KDA, MoE, and Attention Residuals

A technical breakdown of Kimi K3's three core innovations: Kimi Delta Attention, Stable Latente mixture of experts, and attention residuals explained clearly.

Meituan's LongCat 2.0: Open Source AI With 1M Token Context

Dev Kapoor1 month ago

AI. Dev Kapoor1 month ago

Meituan's LongCat 2.0: Open Source AI With 1M Token Context

Meituan's LongCat 2.0 is a 1.6 trillion parameter open-source AI with a 1M token context window. Here's what developers need to know about it.

A smiling man in a beige shirt appears against a yellow-orange circular background, with text about routing algorithms and…

Decoding MoE: Token Routing with a Twist

Yuki Okonkwo7 months ago

AI. Yuki Okonkwo7 months ago

Decoding MoE: Token Routing with a Twist

Explore how Mixture of Experts models use token routing to optimize AI model efficiency and performance.