[Hiring] Mid-Senior Databricks Engineer @Arting Digital Private Limited

Role Description

This is a remote position.

Position: Mid-Senior Databricks Engineer

Location: Remote

Experience: 3–6 Years

Employment Type: Full-Time

Qualifications

• 3–6 years of experience in Data Engineering.
• Strong knowledge of SQL, Python, and PySpark.
• Hands-on experience with Databricks and Apache Spark.
• Understanding of Spark optimization, partitioning, and shuffling.
• Experience with Delta Lake, Z-Ordering, and VACUUM.
• Experience building ETL pipelines using Delta Live Tables (DLT).
• Knowledge of Apache Airflow or Azure Data Factory.
• Understanding of data governance and data quality.

Requirements

• Develop and maintain scalable data pipelines using Databricks, Python, and PySpark.
• Build and automate ETL workflows using Delta Live Tables (DLT).
• Optimize Spark jobs through partitioning, shuffling, and performance tuning.
• Implement Delta Lake optimization techniques, including Z-Ordering and VACUUM.
• Manage data orchestration using Airflow or Azure Data Factory.
• Ensure data quality, governance, and pipeline performance.
• Collaborate with cross-functional teams to deliver efficient data engineering solutions.