[Hiring] Data Engineer @Anteriad

Role Description

As our Data Engineer, you will support our custom data platforms. This role focuses on building, maintaining, and continuously evolving data ingestion and transformation pipelines using:

• Azure Data Factory
• Databricks
• Emerging Microsoft Fabric capabilities

You will play a key role as we modernize and migrate SQL Server–based workloads currently running on Azure virtual machines to Azure-native data services.

You will stand out if you have hands-on experience using Azure data tools as well as SQL Server experience.

Each custom data platform is actively maintained and enhanced over time to support new and evolving requests from campaign services and analytics teams. You must be comfortable working on ongoing, parallel initiatives, balancing modernization work with continuous delivery of incremental enhancements.

Qualifications

• Hands-on experience building or supporting pipelines in Azure Data Factory.
• Experience developing data transformations in Databricks, preferably using SQL within notebooks.
• Experience working with Azure Data Lake Storage or similar cloud storage platforms.
• Familiarity with modern data lake and analytics concepts, including medallion architecture.
• Experience with SQL Server, including writing, maintaining, and understanding complex SQL scripts and stored procedures.
• Experience using Git / Azure DevOps for source control.
• Experience with Python for use in Databricks notebooks and supporting data pipeline operations.
• Ability to follow established architectural patterns and implementation standards.
• Strong analytical, troubleshooting, and communication skills.

Requirements

• Build and maintain data ingestion pipelines using Azure Data Factory (ADF) to move data into the corporate Azure Data Lake and related platforms.
• Develop and enhance data transformation logic in Databricks notebooks, primarily leveraging SQL-based transformations in existing SQL scripts and stored procedures.
• Support a medallion data architecture (Bronze, Silver, Gold) with Silver and Gold datasets stored primarily in Parquet format.
• Assist in migrating SQL Server–based processes (scripts, stored procedures, SSIS logic) running on Azure VMs into Azure-native pipelines and Databricks transformations.
• Continuously update and enhance existing database solutions and systems based on new and evolving requirements from marketing campaign activation and analytics teams, ensuring data remains accurate, timely, and fit for purpose.
• Manage multiple concurrent initiatives, including modernization efforts and ongoing enhancement requests, while maintaining platform stability and data quality.
• Configure Azure Data Factory pipelines to orchestrate Databricks notebooks and Fabric-related processes as applicable.
• Use Azure DevOps for source control, versioning, and deployment of Azure Data Factory, Databricks, and Fabric assets.
• Test, monitor, and troubleshoot pipelines to ensure data reliability, performance, and quality.
• Collaborate with senior engineers, project managers, campaign services, analytics, and platform teams to deliver ongoing data solutions.
• Participate in the adoption and use of Microsoft Fabric, understanding how Fabric components (e.g., OneLake, Lakehouse, Warehousing) integrate with existing Azure and Databricks workflows.
• Maintain technical documentation for data pipelines, transformations, and datasets.

Benefits

• Comprehensive medical (choice of 3 plans), dental and vision coverage.
• Company paid short-term disability, long term disability and life insurance.
• Optional supplemental life, accident and critical illness insurance plans.
• 401K with company match.
• Flexible PTO and generous holiday schedule.
• Fully paid primary caregiver leave (12 weeks) & parental bonding leave (2 weeks).