Role Description
The Data & Operations AI Specialist serves as the Level 3 technical lead for Artificial Intelligence and Data Platform estate. You will be responsible for the architecture, engineering, and advanced troubleshooting of AI infrastructure, data pipelines, and MLOps lifecycles across a multi-cloud environment (Azure and OCI).
Responsibilities
-
AI Infrastructure & Platform Engineering
-
Design & Architecture: Maintain the monitoring architecture for AI/ML platforms and configure advanced dashboards in Grafana and Azure Monitor.
-
Environment Governance: Manage Azure Machine Learning (AML) workspace configurations, compute targets, and Databricks cluster lifecycles (including runtime versions and platform patching).
-
Resource Optimization: Oversee GPU resource allocation, reserved capacity, and cost-performance optimization to align with FinOps goals.
-
Security Integration: Ensure all AI services utilize private endpoints, VNET integration, and RBAC controls to protect sensitive citizen data.
-
Data Pipeline & ETL Management
-
Pipeline Engineering: Own the design, optimization, and remediation of Azure Data Factory (ADF) and Synapse pipelines.
-
Advanced Troubleshooting: Resolve complex bottlenecks related to authentication failures, data format changes, and ETL performance.
-
SOP Leadership: Author step-by-step Standard Operating Procedures (SOPs) for the L1 NOC team to handle routine monitoring and first-line triage.
-
MLOps & Model Lifecycle
-
Automation: Implement CI/CD pipelines for model training, testing, and deployment to AML endpoints.
-
Model Reliability: Configure data drift detection thresholds and automated retraining triggers.
-
Recovery Operations: Develop self-healing scripts and automated recovery runbooks for critical AI workflows.
-
Governance & Compliance
-
Audit Management: Implement and maintain audit logging for all AI decisions and model outputs, ensuring logs flow to the SIEM/vSOC.
-
Regulatory Alignment: Conduct quarterly AI governance reviews to ensure compliance with NESA standards and data privacy guidelines.
Qualifications
-
AI/ML Platforms: Deep expertise in Azure Machine Learning and Databricks.
-
Data Integration: Proficiency in Azure Data Factory and Synapse.
-
Infrastructure-as-Code (IaC): Experience with Terraform or ARM Templates for reproducible deployments.
-
Observability: Ability to use Dynatrace, Grafana, and Azure Monitor for deep-tier diagnostics.
-
Containerization: Knowledge of AKS, Istio Service Mesh, and KEDA.
-
ITIL Mastery: Strong understanding of ITIL-aligned Incident, Change, and Problem management.
-
Security Mindset: Familiarity with NESA standards and UAE data residency requirements.
-
Technical Writing: Ability to draft complex SOPs and Root Cause Analysis (RCA) documents within 48 hours of an incident.
-
Certifications: Microsoft Azure Data Scientist Associate or Azure AI Engineer Associate is highly preferred.