[Hiring] Senior Technical Architect - Databricks @Quantiphi

Role Description

As Senior Technical Architect - Databricks at Quantiphi, you will solve enterprise data problems and will develop solutions for migration, storage, and processing.

Qualifications

• 12+ years of relevant experience in building cloud-native, hybrid, or multi-cloud solutions including Databricks and AWS.
• Experience with Databricks implementations - developing data pipelines using PySpark.
• Experience in Databricks Workspaces, Notebooks, Delta Lake, APIs.
• Expertise in production grade solutions using:
  • AWS (Redshift, S3, Glue, Lambda, Airflow)
  • PySpark
  • Python
  • Data pipelines using Glue/Airflow/Sagemaker.
• Hands-on experience in working on large cloud-based migration workloads involving SQL and NoSQL Databases.
• Experience with migration of databases to AWS.
• Strong ability in creating roadmaps and architecture to execute migration workloads on AWS.
• Exposure to ETL tools and Data warehouse.
• Experience in SQL and Query optimisation.
• Proficient in SQL-based technologies (MySQL, Oracle DB, SQL Server etc.).
• Experience in creation and maintenance of data dictionaries, metadata repositories, and data lineage documentation.
• Experience building and supporting large-scale systems in a production environment.
• Strong skills to mentor and manage teams of junior and senior data engineers and leading end to end delivery of technical workloads.
• Experience using Github/codecommit for developmental activities.
• Experience designing and optimizing orchestration frameworks using Apache Airflow / Amazon MWAA including DAG optimization, dependency management, retries, monitoring, and operationalization.

Requirements

• Experience with AWS services - S3, Redshift, Secrets Manager.
• Exposure to SageMaker Unified Studio (SMUS) and modern business catalog/governance implementations for metadata management, data discovery, and governed analytics access.
• Experience in implementing data integration projects using ETL.
• Experience in using Airflow or Step Functions for orchestration.
• Exposure to IaC tools like Terraform and to CI/CD tools.
• Prior experience in migrating (on-prem to cloud) and processing large amounts of data.
• Experience in implementing data lake and data warehouse on cloud.
• Experience in implementing the industry's best practices.

Benefits

• Join one of the world’s fastest-growing AI-first digital engineering companies and make a real impact at scale.
• Lead and collaborate with a high-energy team of talented, driven individuals solving complex, meaningful challenges.
• Work with Fortune 500 companies and disruptive innovators in a research-driven environment with 60+ patents.
• Stay ahead of the curve by gaining hands-on experience with cutting-edge AI, ML, data, and cloud technologies while continuously upskilling.
• If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!