Senior Machine Learning Engineer, LLM Inference Optimization
Nebius — London, England, United Kingdom · In-Person · Senior · $100k - $150k/yearly
Role description
Role Overview
As a Senior Machine Learning Engineer on the Applied AI team, you will own model and endpoint optimization from model artifacts through production deployment, focusing on improving latency, throughput, memory efficiency, GPU utilization, and cost per token while maintaining model quality and reliability.
What You Will Do
Your responsibilities will include owning optimization work for specific model families, customer endpoints, or serving backends, running engine comparisons, debugging model quality or performance regressions, and optimizing LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.
Why It Might Be a Fit
This is a hands-on role in which you will work on complex optimization projects, diagnose difficult serving problems, and deliver measurable improvements in production, collaborating with kernel and platform engineers to evaluate serving configurations and resolve performance and quality regressions.
Requirements
- Strong Python and PyTorch engineering skills
- Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems
- Practical knowledge of at least one modern inference stack such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent internal systems
- Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory bandwidth, batching, parallelism, and long-context serving
- Ability to reason quantitatively about latency, throughput, quality, utilization, and cost tradeoffs
- Strong communication skills and ability to collaborate with research, kernel, infrastructure, product, and customer teams
Benefits
- Competitive compensation
- Career growth and learning opportunities
- Flexibility and ownership
- Collaborative and innovative culture
- Opportunity to work on impactful AI projects
- International environment and talented teams