Role Description
We are looking for an experienced Engineering Manager with strong hands-on expertise in AWS and GCP Data Engineering to lead a large-scale AWS-to-GCP data platform migration. The ideal candidate will have strong experience designing enterprise data platforms on AWS and migrating them to Google Cloud Platform (GCP). The role requires a combination of technical architecture, hands-on engineering, migration leadership, team management and stakeholder management.
Key Responsibilities
-
AI/ML, Generative AI & MLOps
-
Design and implement AI/ML and Generative AI solutions on GCP using Vertex AI, BigQuery, Cloud Storage, Dataflow, Pub/Sub, Cloud Run, and related GCP-native services.
-
Build production-grade machine learning pipelines for data preparation, model training, validation, evaluation, deployment, monitoring, retraining, and lifecycle management.
-
Develop Generative AI and Retrieval-Augmented Generation (RAG) solutions, including enterprise search, document intelligence, AI assistants, summarization, semantic search, embeddings, vector search, and knowledge-management applications.
-
Design scalable ingestion, transformation, chunking, embedding, indexing, and retrieval pipelines for structured and unstructured enterprise data.
-
Implement MLOps practices using Vertex AI Pipelines, Model Registry, model endpoints, Terraform, GitHub, Cloud Build, and CI/CD pipelines.
-
Establish standards for model versioning, experiment tracking, data and feature validation, automated testing, deployment approvals, rollback, and environment promotion.
-
Implement monitoring for model performance, data drift, latency, reliability, inference cost, response quality, retrieval accuracy, and GenAI risks such as hallucination and prompt injection.
-
Ensure responsible AI, data privacy, security, governance, access control, auditability, and human-review processes are incorporated into AI/ML and GenAI solutions.
-
Partner with Data Science, Analytics, Product, BI, Security, and US-based stakeholders to identify, prioritize, and deliver high-value AI/ML and GenAI use cases.
-
GCP Data Platform Architecture
-
Architect and implement scalable enterprise data platforms on GCP.
-
Design Data Lake and Lakehouse architectures using GCS and BigQuery.
-
Define Bronze, Silver and Gold/Atomic data layers.
-
Design scalable data ingestion, transformation and consumption frameworks.
-
Establish standards for data modeling, partitioning, clustering and storage.
-
Design multi-tenant and multi-location data architectures.
-
Define schema-on-read and schema-on-write strategies.
-
AWS Data Platform Expertise
-
Analyze and optimize existing AWS data platforms before migration.
-
Work with:
-
Amazon S3
-
AWS Glue
-
AWS Glue Data Quality
-
Amazon Redshift / Redshift Serverless
-
Amazon Athena
-
AWS Step Functions
-
AWS DMS
-
AWS Lake Formation
-
Understand existing AWS ETL/ELT pipelines, data models, workloads and dependencies.
-
Identify equivalent or improved GCP services for each AWS workload.
-
Prepare technical mapping and migration plans between AWS and GCP services.
-
GCP Streaming & Real-Time Data Engineering
-
Architect real-time data pipelines using:
-
Google Pub/Sub
-
Dataflow / Apache Beam
-
BigQuery
-
Cloud Storage
-
Design high-volume event ingestion, enrichment and transformation pipelines.
-
Implement event-driven architectures and appropriate delivery guarantees.
-
Optimize streaming pipelines for latency, throughput and scalability.
-
Design BigQuery streaming ingestion patterns.
-
Implement monitoring, logging and alerting for real-time workloads.
-
ETL / ELT & Data Processing
-
Design and implement scalable batch and real-time ETL/ELT pipelines.
-
Migrate AWS Glue-based pipelines to appropriate GCP services.
-
Develop transformation frameworks using:
-
Python
-
PySpark
-
SQL
-
Dataflow / Apache Beam
-
BigQuery
-
dbt
-
Design CDC pipelines and real-time ingestion patterns.
-
Build orchestration workflows using Cloud Composer / Airflow.
-
Optimize data processing jobs and query performance.
-
AWS to GCP Migration Leadership
-
Lead the end-to-end migration of enterprise data platforms from AWS to GCP.
-
Assess existing AWS architecture, data pipelines, workloads, dependencies and operational processes.
-
Define the target-state GCP architecture and migration roadmap.
-
Develop migration strategies for:
-
Amazon S3 β Google Cloud Storage
-
Amazon Redshift β BigQuery
-
AWS Glue β Dataflow / Dataproc / BigQuery
-
AWS Step Functions β Cloud Composer / Workflows
-
AWS DMS β GCP-native CDC solutions
-
Amazon Athena β BigQuery
-
Identify opportunities to modernize AWS workloads rather than performing a simple lift-and-shift migration.
-
Define migration phases, technical dependencies, risks and rollback strategies.
-
Lead architecture reviews and technical design discussions.
-
Data Modeling & BigQuery
-
Design enterprise data models for analytics and reporting.
-
Define dimensional, normalized and denormalized data models.
-
Develop multi-tenant data structures.
-
Design BigQuery partitioning and clustering strategies.
-
Optimize BigQuery SQL and query execution.
-
Design data models supporting both real-time and batch workloads.
-
Work closely with BI and Analytics teams to create scalable consumption models.
-
Data Governance, Security & Quality
-
Establish data governance and data quality standards across the GCP platform.
-
Implement automated data quality checks and validation frameworks.
-
Establish data lineage, metadata and ownership standards.
-
Ensure appropriate security controls across all GCP data layers.
-
Implement:
-
IAM
-
Least-privilege access
-
Encryption
-
Service accounts
-
Network security
-
Data access policies
-
Work with governance and security teams to ensure compliance requirements are met.
-
Experience with Dataplex, Data Catalog and data lineage is preferred.
-
DevOps, Infrastructure & Automation
-
Lead infrastructure automation using Terraform.
-
Build repeatable and secure GCP infrastructure deployments.
-
Implement CI/CD pipelines for data engineering workloads.
-
Work with:
-
Terraform
-
Git
-
GitHub
-
Cloud Build
-
CI/CD pipelines
-
Automate data pipeline deployment, testing and infrastructure provisioning.
-
Establish Dev, QA, UAT and Production deployment standards.
-
Performance & Cost Optimization
-
Lead performance optimization initiatives across GCP data workloads.
-
Optimize:
-
BigQuery query performance
-
Partitioning and clustering
-
Dataflow pipelines
-
Spark workloads
-
Cloud Storage
-
Streaming workloads
-
Analyze AWS workloads and determine the most cost-effective GCP architecture.
-
Develop cloud FinOps and cost optimization strategies.
-
Establish performance benchmarks and SLAs for critical workloads.
-
Engineering Management & Team Leadership
-
Lead and mentor a team of Data Engineers, Senior Data Engineers and Technical Leads.
-
Provide technical direction and establish engineering standards.
-
Conduct architecture and code reviews.
-
Define technical roadmaps and engineering priorities.
-
Break complex migration requirements into actionable deliverables.
-
Track engineering progress, risks, dependencies and delivery milestones.
-
Promote best practices around coding, testing, CI/CD, security and documentation.
-
Mentor engineers on GCP, data architecture and modern data engineering practices.
-
Stakeholder & Client Management
-
Act as the primary technical point of contact for US-based stakeholders.
-
Work closely with Business, Product, Data Science, BI and DevOps teams.
-
Translate business requirements into scalable technical solutions.
-
Present architecture decisions, migration strategies and technical roadmaps.
-
Communicate technical risks, dependencies, timelines and trade-offs.
-
Collaborate with business teams to define operational and analytical KPIs.
Qualifications
-
15+ years of experience in GCP Data Engineering, Data Architecture, Cloud Engineering, AI/ML Engineering, or related technology leadership roles.
-
5+ years of strong hands-on GCP Data Engineering Experience.
-
3+ years of strong hands-on AI/ML & Gen AI Experience.
-
Proven experience delivering AWS-to-GCP migration projects.
-
Strong experience designing enterprise Data Lake and Lakehouse platforms on GCP.
-
Strong hands-on experience with BigQuery, Google Cloud Storage, Dataflow, Pub/Sub, Cloud Composer, Dataproc, IAM, and Terraform.
-
Experience migrating AWS data workloads, pipelines, and platforms to GCP.
-
Strong knowledge of AWS and GCP service mapping, migration patterns, modernization strategies, and cloud architecture best practices.
-
Experience designing, building, and deploying AI/ML solutions on GCP using Vertex AI.
-
Hands-on experience with Generative AI, LLM-based applications, RAG architectures, embeddings, vector search, prompt engineering, and enterprise AI assistants.
-
Strong understanding of MLOps, including model training, model registry, CI/CD/CT, model deployment, monitoring, retraining, governance, and rollback strategies.
-
Experience implementing secure and responsible AI solutions, including data privacy, model evaluation, access controls, auditability, and governance.
-
Expert-level SQL and strong Python and PySpark skills.
-
Strong data modeling, data warehousing, batch processing, and real-time data engineering experience.
-
Experience with Terraform, Git, GitHub, Cloud Build, CI/CD pipelines, and infrastructure automation.
-
Experience managing and mentoring data engineering and cross-functional technical teams.
-
Strong communication skills with experience working with US-based stakeholders.
Preferred Qualifications
-
Google Cloud Professional Data Engineer certification.
-
Google Cloud Professional Machine Learning Engineer certification.
-
Experience with Vertex AI Agent Builder, Vertex AI Search, Gemini models on Vertex AI, or enterprise Generative AI platforms.
-
Experience with dbt, Apache Airflow, Kafka, Apache Spark, Kubernetes, Cloud Run, and API-driven architectures.
-
Experience with Dataplex, Data Catalog, data lineage, metadata management, data governance, master data management, and data-quality frameworks.
-
Experience supporting enterprise or regulated environments with strong data privacy, security, compliance, audit, and governance requirements.
Benefits
-
Lead AWS-to-GCP cloud transformation & AIML GenAI initiatives.
-
Work on Data Lakehouse and analytics modernization.
-
Flexible remote work.
-
Exposure to global customers.
-
Collaborative, innovation-driven culture.
-
Continuous learning and certification.
-
Lead transformative AI/ML & GCP innovations as Head of Engineering at Naveera Tech.
-
Join a global team to revolutionize data into business value.
-
Remote role, USA-based.
-
Apply today!