[Hiring] Senior Site Reliability Engineer & Automation Engineer @Bitdeer Technologies Group

Role Description

NeoCloud is building an AI-operated GPU cloud — and because it is a customer-facing cloud service, reliability is the product. Tenants run mission-critical training, fine-tuning, and inference workloads on our GPU infrastructure and trust us with their SLAs. In this role you own the reliability of the customer-facing GPU cloud service end-to-end: from tenant onboarding and service provisioning, through workload execution, incident response, and post-incident recovery. You are the SRE who stands between raw infrastructure and the customer's experience — designing the observability, automation, and operational practices that make a 10,000-GPU cloud feel simple and dependable to the tenants who depend on it.

What You'll Own

• End-to-end reliability of the customer-facing GPU cloud service — availability, job completion, provisioning latency, and tenant experience.
• Production Kubernetes clusters optimized for GPU workloads at scale (100–10,000 GPUs) as the runtime substrate for customer workloads.
• Nvidia GPU operator, device plugin, MIG configuration, GPU time-slicing, and multi-tenant GPU allocation policies.
• Topology-aware scheduling: GPU locality, NVLink domain awareness, network rail affinity — placing customer jobs on the right hardware.
• Customer & tenant lifecycle: onboarding, quota management, isolation enforcement (namespaces, network policies, RBAC, resource quotas, pod security), and offboarding/reclamation.
• Bare-Metal-as-a-Service (BMaaS): automated provisioning, tenant handoff, lifecycle, and reclamation.
• SLIs/SLOs/SLAs for the customer cloud service: cluster availability, job completion rates, provisioning latency, API availability.
• Incident management with customer communication: runbook automation, escalation, customer-facing status updates, and post-incident reviews.
• Monitoring & observability stack: Prometheus, Grafana, Alertmanager, PagerDuty — tenant-aware dashboards and alerting.
• GPU node failure handling: automated detection, drain/cordon/taint, and workload rescheduling — minimizing customer-visible impact.
• Infrastructure-as-code: Terraform providers/modules, Helm, and GitOps (ArgoCD/Flux) across GPU clusters.
• Customer-facing operational readiness: service documentation, tenant runbooks, capacity planning, and support tiering.

Customer-Facing Ownership

• You are accountable for the customer's reliability experience — when a tenant's job fails or a node drops, you own the detection, remediation, and communication loop.
• Define and publish customer-facing SLAs/SLOs and drive error-budget-based prioritization between feature work and reliability.
• Partner with customer success / support to close the feedback loop between customer-reported issues and systemic improvements.
• Build self-service observability that lets customers answer their own questions — status, quota, job health — reducing support load.

Feed the AIOps Substrate

• The remediation-actuator and workflow engine land here — you make the control plane safe for automated action.
• Your CRDs and runbooks are the schema the platform's predictors and remediators write against.
• Every human intervention you do this quarter becomes an autonomous workflow next quarter — turning customer-impacting incidents into self-healing events.

What Success Looks Like in Year 1

• Customer-facing GPU cloud service SLAs published and met — availability, job completion, provisioning latency.
• Automated drain/reschedule around predicted GPU faults, at scale, without customer-visible impact.
• BMaaS live for external tenants with self-service onboarding.
• MTTD and MTTR for customer-impacting incidents reduced through automation.
• Tenant self-service observability live — customers can see their own job health, quota, and status.

Qualifications

• 5+ years in SRE / cloud operations, with at least 2 years operating GPU workloads at scale.
• Deep understanding of Kubernetes operations and GPU workload management (Nvidia GPU operator, device plugin, MIG, time-slicing, GPU scheduling).
• Experience with topology-aware scheduling and GPU-specific resource management.
• Hands-on experience building multi-tenant cloud platforms with strong isolation guarantees.
• Customer-facing cloud service experience — defining and operating against customer SLAs/SLOs, handling tenant incidents and communications.
• Experience with bare-metal server provisioning and lifecycle automation (Ironic, MAAS, or custom).
• Proficiency in Terraform, Helm, and GitOps workflows (ArgoCD/Flux).
• Strong SRE background: SLI/SLO/SLA frameworks, error budgets, incident management, capacity planning.
• Experience with Prometheus, Grafana, and alerting at scale.
• Strong programming skills in Go or Python for automation / operator development.
• AIOps aptitude — you view the control plane as an execution surface for automated remediation, not just a scheduler.
• Runbook-as-code mindset — every SRE playbook you write should be executable by the platform.

Benefits

• Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws.
• Bitdeer does not discriminate against employees or applicants based on conditions such as race, color, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.