[Hiring] Databricks Engineer @V4C.ai

Role Description

We are seeking a production-focused Senior Data Engineer with exceptional communication skills and a proven track record of delivering reliable data pipelines. In this role, you will have end-to-end production ownership of our data platforms, ensuring high data quality and robust system performance. The ideal candidate prioritizes practical problem-solving and hands-on execution over high-level architecture. You excel at debugging complex data issues and can clearly communicate technical resolutions using a direct, impact-driven approach.

Key Responsibilities

• Pipeline Development: Build, optimize, and maintain robust data pipelines using SQL, dbt, and Python.
• Data Quality & Testing: Implement rigorous data-quality checks and automated testing frameworks within dbt to proactively catch anomalies.
• Production Ownership: Monitor production workflows, manage alerts, and rapidly debug pipeline failures from root cause to permanent fix.
• Data Transformation: Write efficient PySpark transformations to process large-scale datasets within our Lakehouse environment.
• Collaboration & Communication: Translate complex data issues into clear, concise action plans for both technical teammates and business stakeholders.

Qualifications

• SQL & dbt: Expert-level SQL skills and extensive hands-on experience building modular models, snapshots, and custom schema tests in dbt.
• Python: Strong proficiency in writing clean, maintainable Python scripts for data manipulation, automation, and API integrations.
• PySpark: Decent, practical experience utilizing PySpark for data transformation and basic performance tuning (e.g., handling joins, DataFrame API).
• Lakehouse Knowledge: Functional understanding of Delta Lake or similar Lakehouse technologies, including ACID transactions and MERGE operations.

Requirements

• Strong Communication: Exceptional written and spoken communication skills. Ability to explain problems and results directly and concisely.
• Debugging & Problem Solving: Superior troubleshooting skills with a methodical approach to identifying root causes and validating fixes.
• Production Mindset: Experience owning pipelines end-to-end, including CI/CD deployment, monitoring, and handling schema drift.