Senior Machine Learning Engineer, LLM Inference Optimization

Role description

Role Overview

As a Senior Machine Learning Engineer on the Applied AI team, you will own model and endpoint optimization from model artifacts through production deployment, focusing on improving latency, throughput, memory efficiency, GPU utilization, and cost per token while maintaining model quality and reliability.

What You Will Do

Your responsibilities will include owning optimization work for specific model families, customer endpoints, or serving backends, running engine comparisons, and recommending practical serving configurations for specific workloads.

Why It Might Be a Fit

You will work closely with kernel and platform engineers, evaluate serving configurations, resolve performance and quality regressions, and optimize inference for real-world workloads, supported by reproducible benchmarks and safe production rollouts.

Requirements

• Strong Python and PyTorch engineering skills
• Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems
• Practical knowledge of at least one modern inference stack such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent internal systems
• Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving
• Ability to reason quantitatively about latency, throughput, quality, utilization, and cost tradeoffs
• Strong communication skills and ability to collaborate with research, kernel, infrastructure, product, and customer teams

Benefits

• Competitive compensation
• Career growth and learning opportunities
• Flexibility and ownership
• Collaborative and innovative culture
• Opportunity to work on impactful AI projects
• International environment and talented teams