[Hiring] Senior Data Engineer @EIG Services Inc

Role Description

Senior Data Engineers are responsible for designing, developing, and maintaining scalable data platforms and pipelines that align with business objectives and drive operational efficiency. They analyze complex data challenges, architect reliable and secure data systems, and leverage advanced tools and technologies to meet evolving organizational needs.

This role centers on the Databricks ecosystem and Python-based data engineering, with a strong emphasis on building production-grade data pipelines, optimizing data workflows, and enabling analytics and reporting capabilities across the organization.

Key to success in this role is the ability to collaborate across cross-functional teams, apply critical thinking, and translate business requirements into actionable, high-impact data solutions. Familiarity with machine learning pipelines and feature engineering within a data engineering context is a plus.

Essential Duties and Responsibilities

• Design, develop, and maintain scalable, performant data pipelines using Python, PySpark, and Databricks, following Medallion architecture patterns (Bronze, Silver, Gold).
• Build and optimize Delta Lake tables, manage schema evolution, and implement data quality checks across ingestion and transformation layers.
• Develop and maintain Airflow DAGs for pipeline orchestration, scheduling, and monitoring.
• Collaborate with product management and business stakeholders to interpret requirements and define data specifications.
• Create and maintain semantic data layers and curated datasets for BI platforms (e.g., Power BI, QuickSight).
• Analyze trade-offs between commercial and custom-built data solutions, offering technical guidance to decision-makers.
• Troubleshoot and resolve issues across data workflows, pipeline failures, and infrastructure.
• Support machine learning initiatives by building feature pipelines, data preparation workflows, and serving layers within the Databricks platform.
• Ensure compliance with legal, regulatory, and internal governance requirements across data solutions.
• Automate operational processes using CI/CD pipelines and infrastructure-as-code principles.
• Establish and promote best practices for data engineering, pipeline development, and monitoring.
• Partner with teams across the organization to translate business needs into technical data solutions.
• Create and maintain detailed technical documentation including data lineage diagrams, pipeline architecture, and workflow documentation.
• Serve as a subject matter expert in data engineering, pipeline optimization, Databricks platform capabilities, and data architecture.
• Provide mentorship to junior team members, fostering professional growth and collaboration.
• Communicate effectively with leadership and stakeholders regarding project status, risks, and milestones.
• Stay current on emerging data engineering technologies and industry trends to drive continuous improvement and innovation.

Qualifications

• 7+ years of experience in data engineering or related roles.
• Strong proficiency in Python and PySpark for data pipeline development and scripting.
• Deep hands-on experience with Databricks, including Delta Live Tables (DLT), Databricks Workflows, Unity Catalog, and SQL Warehouses.
• Strong knowledge of Medallion architecture (Bronze/Silver/Gold) within Databricks.
• Solid understanding of Lakehouse and Delta Lake concepts, including ACID transactions, time travel, and schema enforcement.
• Proficiency in Apache Airflow for workflow orchestration and scheduling.
• Strong knowledge in relational databases, SQL query optimization, and multiple database systems.
• Experience creating semantic data layers for BI platforms (e.g., Power BI, QuickSight).
• Experience combining internal and external data sources for in-depth industry and business analyses.
• Understanding of cloud computing platforms and integration with on-premise solutions.
• Familiarity with CI/CD pipelines, version control (Git), and infrastructure-as-code practices.
• Experience supporting ML pipelines and feature engineering within a data platform context is a plus.
• Strong understanding of testing methodologies, including data validation, pipeline testing, and automation.
• Experience working in regulated industries (e.g., insurance, banking, finance, healthcare).
• Databricks certification preferred (e.g., Databricks Certified Data Engineer Associate or Professional).

Requirements

• Outstanding analytical and problem-solving skills with the ability to simplify complex tasks and absorb new information rapidly.
• Strong written and verbal communication skills, with the ability to convey technical information clearly to both technical and non-technical audiences.
• Highly self-motivated, able to take initiative and manage time effectively to meet deadlines and deliverables.
• Strong attention to detail with the ability to work independently with minimal supervision.
• Ability to manage multiple projects simultaneously in a fast-paced, evolving environment.
• Exceptional research skills with the ability to gather, analyze, and synthesize complex information quickly.
• Passion for delivering exceptional quality and continuous learning of new technologies and frameworks.
• Strong listening skills and the ability to build positive relationships with colleagues, stakeholders, and customers.
• Commitment to creating an inclusive, team-oriented environment that aligns with company culture and values diversity.

Work Environment

• Remote: This position is a fully remote (work‑from‑home) opportunity.
• Candidates must be currently located in the United States and authorized to work without sponsorship.
• A suitable home workspace that is private and free from distractions is required.
• Expected Work Hours: Work schedules are designed to meet both the needs of the role and the organization and may be adjusted as business needs evolve.
• Travel: Possible travel to off‑site locations may be required for meetings, as needed.

Salary Range

$90,000 - $135,000 and a comprehensive benefits package, please follow the link to our benefits page for details!