← Back to stories
Gadgets

AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU

AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances, with up to 48 concurrent invocations sharing one model per GPU, vLLM and SGLang base images that need no handler code, and 90-minute asynchronous timeouts. Billing is the instance price plus a 9.75 percent fee; first hands-on tests report an eight-minute publish and no response streaming.

You're reading a preview. The full article is published by InfoQ on their website.

Read the full story on InfoQ