Artificial Intelligence
Author: Andy Peng
Yiming Peng is a Senior Software Development Engineer at AWS Containers. He is the founding engineer of AWS App Runner and now the tech lead of Data Plane, with a focus on providing customers seamless Auto Scaling, Request-Routing, Load Balancing, Networking, Observability etc. capabilities. Yiming closely works on App Runner, ECS Fargate. He is passionated in Cloud-Native, Container Compute, Serverless and Open-Source. You can find him in Github & Twitter @pymhq.
Parallelize speculative decoding with P-EAGLE on Amazon SageMaker AI
This post walks you through how to use P-EAGLE directly within Amazon SageMaker AI. It will demonstrate how to select a compatible model from the SageMaker JumpStart catalog, configure the parallel drafting specifications, and deploy a highly optimized real-time SageMaker AI endpoint to accelerate your generative AI applications.