Run 3–10 Parallel Gemma 4 Instances Locally: Hardware & Setup Guide

What do you need to run 3, 5, or even 10 parallel instances of Gemma 4 locally?

Google has open-sourced a demo in google-gemma/cookbookgoogle-gemma/cookbook/tree/main/apps/concurrent that makes it possible to run multiple models side-by-side on your hardware.

Gemma 4 26B A4B can easily handle 10+ parallel requests on a MacBook Pro M4 Max at a speed of 18 tokens per second for each request.