[Hiring] Senior Data Engineer @Quartile

Role Description

The Senior Data Engineer owns the substrate end to end: ingestion from messy third-party APIs, modeling and governance in Unity Catalog, the serving layer agents query at runtime, and the health monitoring that tells us something broke before a CSM finds out in a client meeting.

• Design, build, and operate ingestion pipelines from advertising and business platforms (Amazon Ads, Google Ads, Walmart Connect, Criteo, Salesforce, and internal services) via REST APIs, Delta Sharing, object storage, and event streams.
• Model curated data layers in Delta Lake under Unity Catalog: medallion architecture, incremental and CDC patterns, clear contracts between layers, lineage, and row/column-level governance.
• Build the serving layer for AI: SQL views, Unity Catalog functions, and low-latency stores exposed to agents as tools through the Model Context Protocol (MCP) behind Azure API Management.
• Own data quality and health monitoring as code: freshness, volume, schema-drift and business-rule expectations, with alerting wired into Grafana.
• Ship everything through Terraform and CI/CD — Databricks and Azure resources, jobs, permissions, and environment promotion from dev to prod.
• Own performance and cost: warehouse and cluster sizing, partitioning and liquid clustering, job orchestration in Databricks Workflows / Lakeflow.
• Work fluently with AI coding agents as part of your daily engineering practice.
• Collaborate with cross-functional teams — AI engineers, software engineers, and product.
• Instrument what you ship with OpenTelemetry and structured logging.
• Document and maintain the codebase, ensuring code quality and adherence to best practices.

Qualifications

• Strong software engineering fundamentals in Python and advanced SQL.
• Deep hands-on experience with Databricks: Spark/PySpark, Delta Lake, Unity Catalog, Workflows, and SQL warehouses.
• Proven experience designing data pipelines and data models in production.
• Experience integrating third-party APIs at scale.
• Familiarity with cloud platforms (we run on Azure — ADLS, Container Apps, Key Vault, API Management, Entra ID).
• Experience with Infrastructure as Code (Terraform) and CI/CD pipelines.
• A working definition of data quality: tests, expectations, monitoring.
• Practical experience using AI coding assistants/agents in real engineering work.
• Excellent problem-solving and analytical skills.
• Bachelor's degree or higher in Computer Science, Information Technology, or a related field — or equivalent practical experience.

Requirements

• Experience serving data to LLM-powered applications.
• Experience with operational stores alongside the lakehouse.
• Observability stacks: OpenTelemetry, Grafana, Loki, structured logging.
• Delta Sharing for secure data exchange with clients and partners.
• Domain background in retail media, digital advertising, or marketplace data.
• Databricks or Azure certifications.

Benefits

• This is a PJ contract based in Brazil.