Staff Software Engineer, Inference

Role description

Role Overview

We're looking for a Staff Software Engineer to lead the development of our inference platform. You'll define and drive multi-service, cross-team architecture, and lead complex design initiatives. You'll also partner with Product, Orchestration, Networking, Storage, and Hardware teams to deliver a cohesive inference platform.

What You Will Do

As a Staff Software Engineer, you'll lead the development of advanced inference optimizations, including micro-batching and dynamic scheduling, speculative decoding, KV-cache reuse and memory optimization. You'll also drive measurable improvements in P95/P99 latency, throughput, and GPU utilization across the platform.

Why It Might Be a Fit

We're looking for someone with ~8–12+ years of experience building large-scale distributed systems or cloud platforms. You'll need to have a proven track record of leading cross-team technical initiatives at scale, strong coding skills in Go, Python, or C++, and deep expertise in Kubernetes at production scale.

Requirements

• ~8–12+ years of experience building large-scale distributed systems or cloud platforms
• Proven track record of leading cross-team technical initiatives at scale
• Strong coding skills in Go, Python, or C++
• Deep expertise in Kubernetes at production scale
• Bachelor’s degree in Computer Science, Engineering, or a related technical field

Benefits

• Family-level Medical Insurance
• Family-level Dental Insurance
• Generous Pension Contribution
• Life Assurance at 4x Salary
• Critical Illness Cover
• Employee Assistance Programme
• Tuition Reimbursement