Senior ML Systems Engineer, Large-Scale Model Training & RL Infrastructure

Role description

Role Overview

The Senior ML Systems Engineer owns substantial training or RL infrastructure components end to end, building and maintaining distributed training infrastructure for SFT, continued pretraining, preference optimization, and RL workloads.

What You Will Do

The role involves integrating and extending frameworks, implementing and debugging parallelism strategies, building reliable rollout, reward model serving, replay/data buffer, checkpointing, evaluation, and experiment orchestration components for RL training.

Why It Might Be a Fit

The ideal candidate has strong Python and PyTorch engineering skills, hands-on experience with distributed model training, and a practical understanding of transformer training bottlenecks and memory pressure.

Requirements

• Strong Python and PyTorch engineering skills
• Hands-on experience with distributed model training, large-scale ML systems, or GPU cluster workloads
• Practical understanding of transformer training bottlenecks, memory pressure, gradient/optimizer state, communication overhead, and checkpointing
• Experience debugging production or research training jobs across multiple GPUs or nodes
• Ability to reason quantitatively about throughput, utilization, memory, reliability, cost, and research velocity
• Strong communication skills and ability to collaborate with researchers, ML engineers, platform engineers, and leadership

Benefits

• 100% company-paid medical, dental, and vision coverage for employees and families
• Up to 4% company match with immediate vesting for 401(k) plan
• 20 weeks paid parental leave for primary caregivers, 12 weeks for secondary caregivers
• Up to $85/month remote work reimbursement for mobile and internet
• Company-paid short-term, long-term, and life insurance coverage