[Hiring] Lead Data Platform Engineer @EPAM Systems

Role Description

We are seeking a Lead Data Platform Engineer to design, build, and optimize scalable Databricks data platforms that power analytics, reporting, and AI initiatives. You will own modern pipelines, governance, and high-quality data products in the cloud.

• Design scalable data platform architectures on Databricks to support analytics and AI use cases
• Build end-to-end data pipelines using Lakeflow DLT, Auto Loader, and Medallion Architecture
• Implement data governance with Unity Catalog, including access controls, lineage, and metadata standards
• Optimize Apache Spark workloads for performance, reliability, and cost efficiency
• Develop dimensional data models and curated datasets for reporting and downstream consumption
• Deliver AI/ML-ready datasets and enable feature engineering or GenAI/RAG data pipelines
• Automate deployments with CI/CD using GitHub Actions and Databricks Asset Bundles
• Integrate AWS services such as Amazon S3, IAM, KMS, DMS, and Aurora PostgreSQL into data solutions
• Establish data quality checks, monitoring, and operational runbooks for production pipelines
• Collaborate with stakeholders to translate requirements into well-defined data products and SLAs
• Review code and set engineering standards to improve maintainability and delivery speed

Qualifications

• 5+ years of data engineering or analytics engineering experience
• 1+ years of leadership experience driving platform standards and technical decisions
• Strong project delivery skills owning end-to-end data platform initiatives
• Advanced Databricks skills with Unity Catalog, Delta Lake, and Databricks for Data Analytics
• Strong Apache Spark skills using PySpark and SQL for large-scale processing
• Strong data pipeline skills with Lakeflow DLT, Auto Loader, and Medallion Architecture
• Advanced data modeling skills with FACT/DIM, Star Schema, and SCD Type 2
• Strong data governance skills including metadata management, lineage, and access controls
• Strong AWS skills with Amazon S3, IAM, KMS, DMS, and Aurora PostgreSQL
• Strong CI/CD skills with Git, GitHub Actions, and Databricks Asset Bundles (DABs)
• Strong communication skills for cross-team collaboration and clear technical writing
• English proficiency: B2 Upper-Intermediate

Requirements

• Nice to have Databricks Mosaic AI, Vector Search, Lakebase, or Model Serving experience
• AI/BI (Genie), semantic layers, or natural language analytics experience
• Healthcare or Life Sciences domain knowledge including HIPAA/PHI considerations
• Change Data Capture (CDC) implementation experience
• Serverless data platform experience