[Hiring] Machine Learning Engineer - ML Training Platform @Pluralis Research

Role Description

This role involves implementing a novel substrate for training distributed ML models that work under consumer grade internet connection.

• Design and implement large-scale distributed training systems optimized for heterogeneous hardware operating under low-bandwidth, high-latency conditions.
• Develop and optimize model-parallel training strategies (data, tensor, pipeline parallelism) with custom sharding techniques that minimize communication overhead.
• Optimize GPU utilization, memory efficiency, and compute performance across distributed nodes.
• Implement robust checkpointing, state synchronization, and recovery mechanisms for long-running, fault-prone training jobs.
• Build monitoring and metrics systems to track training progress, model quality, and system bottlenecks.
• Architect resilient training systems where nodes can fail, networks can partition, and participants can dynamically join or leave.
• Design and optimize peer-to-peer topologies for decentralized coordination across non-co-located nodes.
• Implement NAT traversal, peer discovery, dynamic routing, and connection lifecycle management.
• Profile and optimize communication patterns to reduce latency and bandwidth overhead in multi-participant environments.

Qualifications

• Strong experience building and operating distributed systems in production.
• Hands-on expertise with distributed training frameworks (FSDP, DeepSpeed, Megatron, or similar).
• Deep understanding of model parallelism (data, tensor, pipeline parallelism).
• Expert-level Python with production experience (concurrency, error handling, retry logic, clean architecture).
• Strong networking fundamentals: P2P systems, gRPC, routing, NAT traversal, distributed coordination.
• Experience optimizing GPU workloads, memory management, and large-scale compute efficiency.

Requirements

• 5+ years of experience in distributed systems and ML large-scale training.

Benefits

• Equity-heavy compensation with meaningful ownership in a mission-driven company.
• Competitive base salary for senior engineering roles in Australia.
• Visa sponsorship available for exceptional candidates.
• Remote-first with optional access to our Melbourne hub.
• World-class team — team mates were previously at Google, Amazon, Microsoft, and leading startups.