[Hiring] Databricks Engineer @V4C.ai

Role Description

As a Databricks Engineer, you will build and optimize production pipelines on the Databricks Lakehouse, working directly with customers and often Databricks' own field teams. You will work under a senior engineer or architect, taking a defined piece of an engagement and owning it end to end:

• Ingesting from systems that were not designed to be ingested from
• Transforming data through Medallion layers
• Getting data to a state analysts and ML workloads can actually use

You will be client-facing, participating in standups and working sessions, and occasionally explaining to a stakeholder why something is harder than it looked.

Responsibilities

• Build and maintain batch and streaming ETL pipelines using PySpark, SQL, and Delta Lake on Databricks
• Implement Medallion architecture layers
• Deploy and manage data resources on AWS, Azure, or GCP
• Write and automate data validation and quality checks
• Tune Spark jobs and SQL queries for performance and cost
• Structure and clean enterprise datasets for downstream analytics and AI workloads
• Troubleshoot pipeline failures and data quality issues, including in production

Qualifications

• 8+ years in data engineering/architecture or analytics platform development
• Demonstrated ownership of end-to-end architecture for at least two enterprise-scale data platforms, from design through production
• Deep experience migrating legacy data warehouses or ETL estates to a modern cloud platform (Teradata, Informatica, Hadoop, Netezza, or equivalent)
• Strong proficiency in Python and SQL; working knowledge of Apache Spark and distributed processing fundamentals
• Experience with data modeling across dimensional, normalized, and Lakehouse patterns, and the judgment to know when each applies
• Experience with CI/CD and DevOps practices for data platform deployment

Preferences

• Healthcare or life sciences experience preferred but not required
• Databricks Certified Data Engineer Professional or a Platform Architect
• Hands-on Databricks experience at scale: Unity Catalog, Delta Lake, Lakeflow, Asset Bundles
• Experience with MLOps or productionizing machine learning workloads
• Prior professional services, consulting, or customer-facing architecture background
• Experience contributing to pre-sales scoping or technical proposals

Benefits

• Unlimited PTO
• Health Care Plan (Medical, Dental & Vision)
• Retirement Plan (401k, IRA)
• Life Insurance (Basic, Voluntary & AD&D)
• Family Leave (Maternity, Paternity)
• Short Term & Long Term Disability
• Training & Development
• Remote/WFH