[Hiring] Senior Azure Data Engineer @USEReady Technology

Role Description

We are looking for an experienced Senior Azure Data Engineer with strong hands-on expertise in Databricks, Azure Data Factory (ADF), PySpark, Python, and SQL. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data engineering solutions on Azure.

You will work with modern Azure data technologies to build high-performance data pipelines, implement data governance, optimize Databricks workloads, and deliver reliable data platforms that support analytics and business intelligence initiatives.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Azure Databricks and Azure Data Factory (ADF).
• Develop data processing solutions using Python, PySpark, SQL, and Apache Spark.
• Build and maintain Delta Lake tables and scalable data models for analytics and BI workloads.
• Implement and manage Databricks Unity Catalog for data governance, security, access control, and data discovery.
• Optimize Databricks jobs, Spark workloads, SQL queries, and data pipelines for performance, scalability, and cost efficiency.
• Apply Apache Spark internals and tuning techniques to improve distributed data processing performance.
• Develop complex and optimized SQL queries, including efficient joins and processing of large datasets.
• Design and implement ETL/ELT workflows using Azure Data Factory.
• Work with Azure data services including Azure Data Lake Storage (ADLS), Azure Synapse Analytics, and Azure SQL Database.
• Implement data quality checks, validation, reconciliation, monitoring, and error-handling mechanisms.
• Implement data governance, security, access control, and data lineage across data platforms.
• Troubleshoot pipeline failures and performance issues and ensure high availability and reliability of data solutions.
• Collaborate with business stakeholders, data architects, analysts, and engineering teams to understand requirements and deliver effective data solutions.
• Automate manual processes and continuously improve data engineering workflows and operational efficiency.
• Mentor junior data engineers and promote best practices across Azure and Databricks data engineering.

Qualifications

• 6+ years of experience in Data Engineering, with strong experience in Azure-based data platforms.
• Strong hands-on experience with Azure Databricks.
• Expertise in Databricks Notebooks using Python and SQL.
• Strong experience with Databricks Unity Catalog, governance, security, and access control.
• Hands-on experience with Databricks performance optimization and cost/compute optimization.
• Strong experience with Azure Data Factory (ADF) and ETL/ELT orchestration.
• Advanced Python and PySpark programming skills.
• Strong understanding of Apache Spark architecture, internals, and performance tuning.
• Strong SQL skills, including complex queries, query optimization, joins, aggregations, and large-volume data processing.
• Strong understanding of data warehousing concepts and data modeling, including Star and Snowflake schemas.
• Experience with Delta Lake and modern lakehouse architecture.
• Good knowledge of Azure Data Lake Storage (ADLS).
• Experience with Azure Synapse Analytics and Azure SQL Database.
• Understanding of data governance, data security, access management, and data lineage.
• Experience working with large-scale distributed data processing environments.
• Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

• Databricks certifications are a plus.
• Microsoft Azure Data Engineering certifications are a plus.
• Experience working with enterprise-scale Azure data platforms.
• Experience implementing data quality, governance, and security frameworks.
• Experience mentoring or technically guiding junior data engineers.

Education

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field is preferred.