[Hiring] Senior Data Engineer @Hack The Box

Role Description

Let's redefine cyber security expertise standards and connect business - community through highly engaging hacking experiences.

✨The core mission of the Senior Data Engineer:

• Own and evolve data pipelines on GCP.
• Build new pipelines, harden existing ones, and improve data quality.
• Make clean, trustworthy data available across the organization.
• Work end-to-end on streaming and batch pipelines.
• Design ELT/ETL processes on BigQuery and ClickHouse.
• Build real-time pipelines on Pub/Sub and Kafka with Dataflow.
• Orchestrate workflows with Airflow.
• Ensure data is properly cleaned, modeled, and served for analytics, ML training, and online inference.
• Partner with ML engineers on feature pipelines and monitoring data drift.
• Consume and build REST APIs and integrate with third-party SaaS sources.
• Treat infrastructure as code.

Qualifications

• Strong data modelling and warehouse architecture skills.
• Hands-on experience with GCP data services.
• Production experience with streaming pipelines.
• Solid SQL and strong Python skills.
• Experience with ClickHouse or another columnar OLAP engine.
• Workflow orchestration experience with Airflow.
• Comfortable with dbt or equivalent transformation frameworks.
• Experience migrating off legacy warehouses onto cloud-native stacks is a plus.
• Working knowledge of ML in production.
• Docker & Kubernetes experience.
• CI/CD mindset and infrastructure-as-code sensibility.

Requirements

• Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka.
• Drive the migration off Snowflake onto GCP-native stack.
• Own the orchestration layer in Airflow.
• Model data for analytics and ML.
• Partner with ML engineers on feature stores and retraining workflows.
• Capture requirements from stakeholders and translate them into data products.
• Continuously improve data quality, reliability, observability, and cost efficiency.
• Identify new data sources worth acquiring and integrate them cleanly.

Benefits

• Compensation: $140,000-160,000.
• Medical, Dental & Vision (employee coverage 100% paid).
• 401K with employer match.
• Employer-paid Life and AD&D Insurance.
• Supplemental Life Insurance.
• Short-term and Long-term Disability.
• Healthcare and Dependent Care FSA.
• Paid parental leave.
• 25 annual leave days.
• Home Office Allowance.
• Dedicated budget for training and professional development.
• State-of-the-art equipment.
• Full access to the Hack The Box lab offerings.