[Hiring] AWS Lake Formation Architect Specialist @Adastra Corporation

Role Description

We are seeking a Senior AWS Lake Formation Architect Specialist to design, implement, and govern secure, scalable data lake and lakehouse architectures on AWS. The ideal candidate will bring strong hands-on expertise in AWS Lake Formation, cross-account data sharing, and modern open table formats such as Apache Iceberg and Apache Hudi.

In this role, you will lead the design of enterprise-scale data governance frameworks, implement fine-grained access controls, and enable secure data sharing across multiple AWS accounts and business units. You will collaborate with data engineering, security, and business teams to deliver scalable, high-performance, and compliant data solutions.

Responsibilities

• Architect end-to-end data lake and lakehouse solutions using AWS Lake Formation, S3, Glue Data Catalog, Athena, Redshift Spectrum, and EMR.
• Design and implement open table format strategies using Apache Iceberg and Apache Hudi to support ACID transactions, schema evolution, time travel, and efficient upserts and deletes for CDC use cases.
• Implement fine-grained, column-, row-, and cell-level access controls using AWS Lake Formation permissions.
• Design tag-based access control (LF-TBAC) strategies and manage data lake administrator roles.
• Establish data classification, lineage tracking, and auditing using CloudTrail and Lake Formation audit logs.
• Architect secure, governed data sharing across multiple AWS accounts, business units, and external partner organizations using Lake Formation resource links, named resource grants, and LF-tags.
• Design hub-and-spoke or data mesh architectures where a central governance account manages the Glue Data Catalog and grants controlled access to consumer accounts.
• Implement cross-account Iceberg and Hudi table sharing while ensuring metadata consistency and permission enforcement across account boundaries.
• Manage AWS Resource Access Manager (AWS RAM) integration for sharing catalog resources.
• Establish cross-region and cross-organization data sharing patterns using AWS Organizations while maintaining compliance boundaries.
• Troubleshoot cross-account permission issues, including IAM role chaining, trust policies, and Lake Formation grant propagation.
• Select and implement appropriate open table formats, including Iceberg, Hudi, and Delta Lake, based on workload requirements such as streaming upserts, batch CDC, and analytics-heavy read patterns.
• Manage Iceberg and Hudi table maintenance, including compaction, clustering, snapshot expiration, and manifest optimization.
• Integrate Iceberg and Hudi tables with Glue Data Catalog, Athena, EMR (Spark), and Redshift Spectrum for unified querying.
• Design streaming ingestion pipelines using Kinesis, MSK, and DMS to support near-real-time lake updates.
• Ensure architectures meet regulatory requirements such as GDPR, HIPAA, and SOC 2 through encryption, lineage, and access auditing.
• Optimize partitioning, file formats such as Parquet and ORC, and storage tiering for performance and cost efficiency.
• Collaborate with data engineers, data scientists, security and compliance teams, and business stakeholders to translate governance and data-sharing requirements into architecture.
• Lead migrations from legacy data warehouses and on-premises data lakes to AWS lakehouse architectures with transactional table formats.

Qualifications

• 5+ years of experience in data engineering or architecture, including 2+ years of hands-on experience with AWS Lake Formation.
• Strong hands-on expertise in AWS Lake Formation, AWS Glue Data Catalog, Amazon S3, Athena, Redshift, EMR, and IAM.
• Proven experience designing cross-account and cross-organization data-sharing frameworks using resource links, LF-tags, and AWS RAM.
• Practical knowledge of Apache Iceberg and Apache Hudi, including table internals, metadata and manifest structures, compaction strategies, and integration with AWS query engines.
• Hands-on production experience implementing and managing open table formats such as Iceberg or Hudi.
• Proficiency in Infrastructure as Code tools such as CloudFormation, Terraform, or AWS CDK.
• Strong scripting and programming skills in Python, PySpark, and SQL.
• Solid understanding of data security, encryption, access management, and compliance frameworks.
• Experience designing scalable, secure, and governed multi-account AWS data architectures.

Nice to Have

• Familiarity with Delta Lake and its capabilities for comparative architecture decisions.
• Experience implementing data mesh architectures.
• AWS certifications such as AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Professional.
• Experience with streaming data technologies such as Amazon Kinesis, MSK, and DMS.