[Hiring] Lead DevOps/MLOps Engineer @EPAM

Role Description

We are looking for a Lead DevOps/MLOps Engineer to join our ML Engineering team and help build out and harden our on-premises, GPU-backed ML platform. The goal is to enable multiple ML and LLM use cases to run simultaneously, reliably and with low latency, while scaling the platform to a wider user base. You will work within a 25-person Data Science Department that has grown an in-house agentic platform from a small pilot to roughly 500 employees, with further rollout currently limited by compute capacity.

• Build out and harden the on-premise, GPU-backed ML platform to support multiple concurrent ML and LLM workloads.
• Ensure reliability, low latency, and scalability as the user base expands beyond the current ~500 employees.
• Mature the GPU environment to production-grade standards, addressing daily bottlenecks and resource contention.
• Manage and evolve the GPU infrastructure across 12+ NVIDIA H200 servers, including integration with Kubernetes where appropriate.
• Operate and improve the two Kubernetes clusters (production and sandbox) along with platform services such as MLflow and the Agent Gateway.
• Support self-hosted LLM deployments for internal, confidential data use cases.
• Maintain and extend the Ray environment shared by the Data Science and Deep Learning teams for training workloads.
• Enhance monitoring and observability using NVIDIA DCGM exporter, Prometheus, and Grafana.
• Collaborate closely with the Traditional ML, Deep Learning, and Data Analytics teams to align infrastructure with evolving use cases.

Qualifications

• 5+ years of experience in DevOps/MLOps engineering roles.
• At least 1 year of relevant leadership experience.
• Hands-on experience running GPU workloads on Kubernetes or OpenShift in production, ideally on-premises.
• Practical experience deploying and troubleshooting the NVIDIA GPU Operator with a clear grasp of what each component does.
• Understanding of how GPUs differ from CPUs as a scheduled resource, including memory limits, utilisation semantics, and the limitations of default Kubernetes scheduling.
• Knowledge of GPU sharing and isolation methods such as MIG, time-slicing and MPS, and their trade-offs for memory and fault isolation.
• Solid Linux fundamentals including cgroups, namespaces, and systemd resource controls for process-level resource management.
• Skills in Prometheus, Grafana and NVIDIA DCGM, including capturing GPU metrics at process or pod level.
• Working knowledge of the CUDA stack covering drivers, CUDA toolkit, container runtime, and version compatibility.
• Experience deploying with Helm and operating MLOps tools such as MLflow and Airflow on Kubernetes.
• English proficiency at an Upper-Intermediate level (B2) or higher.

Requirements

• Familiarity with LLM serving engines such as vLLM, SGLang or NVIDIA Triton.
• Experience with Ray/KubeRay and GPU-aware schedulers such as Kueue or Volcano.
• Hands-on work with H100/H200-class hardware, GitLab CI and Docker.
• Background in Machine Learning and Python.