[Hiring] Senior Databricks Engineer @CGI

Role Description

We are seeking an experienced Senior Databricks Engineer to serve as a hands-on senior data engineer supporting a greenfield Databricks implementation. The role will design, develop, test, and deploy scalable batch and real-time data engineering solutions while reverse engineering existing AWS-based data processing to identify transformations, business rules, dependencies, and integration requirements that must be preserved or redesigned within the target Databricks architecture.

The Senior Databricks Engineer will deliver prioritized capabilities while helping establish reusable engineering patterns, development standards, governance, deployment practices, and best practices that improve the scalability, reliability, performance, and maintainability of the Databricks platform. This position can be located remotely anywhere in the U.S. Infrequent travel to Dallas, Texas may be required.

Qualifications

• 6+ years of data engineering or software engineering experience, including significant experience designing, developing, and supporting enterprise scale data platforms.
• 3+ years of hands-on Databricks experience developing and operating production data engineering solutions.
• Demonstrated production experience with Databricks Auto Loader.
• Demonstrated production experience with Lakeflow Declarative Pipelines.
• Strong hands-on experience implementing Unity Catalog in a production environment.
• Strong hands-on experience with Delta Lake.
• Strong hands-on experience with Apache Spark, PySpark, and Spark SQL.
• Demonstrated ability to diagnose and optimize Spark workloads.
• Experience developing production Structured Streaming solutions.
• Demonstrated experience designing and implementing Medallion Architecture.
• Experience with Databricks Workflows and deployment as code.
• Strong understanding of distributed data processing, data optimization, scalability, and production pipeline resiliency.
• Experience implementing production grade approaches for data quality, error handling, monitoring, logging, observability, and pipeline recovery.
• Experience with CI/CD, automated deployment practices, Git based source control.

Requirements

• Strong hands-on experience developing and debugging AWS Glue ETL jobs using PySpark and Python.
• Demonstrated ability to analyze unfamiliar and undocumented AWS Glue code.
• Strong working knowledge of Amazon S3.
• Experience with Amazon Redshift.
• Strong understanding of AWS IAM and data access patterns.
• Working knowledge of AWS Glue Crawlers and the Glue Data Catalog.
• Working knowledge of Redshift Spectrum and external S3 backed data access patterns.
• Working knowledge of supporting AWS data, orchestration, monitoring, and security services.
• Demonstrated ability to trace an end-to-end AWS data pipeline across multiple services.

Benefits

• Competitive compensation including profit participation program.
• Comprehensive medical, dental, and vision benefits.
• Basic life and accidental death & dismemberment insurance.
• Matching contributions through 401(k) plan, and CGI share purchase plan.
• Flexibility and paid accrued vacation leave, ranging from 10 to 20 days per year.
• 10 paid holidays per year.
• At least 80 consecutive hours of paid sick/safe leave.
• Paid parental leave, ranging from 20 to 70 consecutive business days.
• Bereavement leave, ranging from 1 to 7 days per year.
• Paid jury duty leave, up to time summoned.
• Learning opportunities and tuition assistance.
• Wellness and well-being programs.