[Hiring] Senior Data Engineer @Arionkoder

Role Description

We’re hiring a Senior Data Engineer to build the structured core of a governed data foundation on Databricks, used by AI agents, applications, and business teams. You own everything that has to be exact:

• Business entities reconciled across systems that use different identifiers
• Facts with their source and validity period
• Business definitions turned into certified tables
• Access control
• The APIs and MCP tools that serve this data with the caller’s permissions

The right person has spent years on Databricks making data correct and consistent, has enforced access control in production, and knows where platform controls stop and application-level enforcement has to start. You work autonomously with business stakeholders and ask the uncomfortable questions about definitions and permissions early.

You’ll work in a small team (solution architect, AI engineer, full-stack engineer, data quality lead, product and project managers) alongside the client’s data platform team. You follow their conventions and hand over what you build. The AI engineer owns the parts that need a model: knowledge from documents, retrieval, agent behavior, and evals. You give them the trusted data those parts depend on. AI tools speed up your modeling, SQL, testing, and documentation every day, and you know exactly when to trust the output and when not to.

Qualifications

• 6+ years of professional data engineering, building production-grade pipelines and data models
• 3+ years hands-on with Databricks in production, including Unity Catalog
• Expert SQL and Python: writes, debugs, optimizes, and refactors pipelines unaided
• Spark/PySpark a plus
• Strong data modeling, including history-aware techniques (SCD Type 2, effective dating, temporal validity), with provenance and lineage designed in from the start
• Data quality and testing discipline: tests and expectations in pipelines (dbt tests, Great Expectations, or similar) and CI/CD for data
• Reconciling data across systems: entity resolution or master data
• Incremental and change-driven processing (CDC, streaming, or incremental frameworks) against freshness targets
• Practical data governance: has implemented access control in production (row-level security, column masking, or role/attribute-based access) and knows where platform-level controls stop and application-level enforcement has to start
• Has built APIs or services that serve data
• Senior judgment: works autonomously, challenges unclear definitions, prioritizes by risk, owns outcomes without micromanagement
• Daily use of AI coding tools (Claude, Cursor, or similar), with the judgment to know when to trust the output
• C1 English; comfortable working fully remote and directly with a client team

Requirements

• Exposing data to applications through APIs or MCPs
• Multi-tenant data isolation for customer-facing products, with security and negative testing
• Finance or CRM data (NetSuite, Salesforce)
• Consolidating data across acquired companies or business units
• Ingesting and governing unstructured content (documents, wikis, contracts) alongside structured data
• Turning business definitions into a shared semantic or metrics layer with business or finance stakeholders
• FinOps / cost attribution on data platforms

What this person will own and do day-to-day

• Ingestion and source strategy: Build and maintain ingestion and ETL pipelines from business systems (ERP, CRM, product databases), operational tools, and document repositories. Build data lakes in Databricks that software, people, and AI agents can all query. Decide per source whether to ingest, index and link, or query live, based on volatility, volume, and which system is the record. Work with raw data in its source-system shape without waiting on upstream cleanup.
• Entities and facts: Implement the entity model defined with the solution architect and resolve identity across systems, reusing existing matching work and sending ambiguous matches to the owner who decides. Model facts with source, validity period, validation status, and documented exceptions. Turn business definitions validated by their owners into certified tables, recording who signed off. Implement authority rules per data item and keep unresolved conflicts visible. Deliver finance numbers that are deterministic, reproducible, and traceable to source records.
• Access and isolation: Implement access policies in Databricks and carry them into derived data. Enforce isolation at the database, tool, and application layers for consumers who aren’t Databricks users, such as external customers. Build isolation tests at the data and API layer, including similarly named customers, and run them as a release gate.
• Structured sources and freshness: Integrate the structured sources each increment needs (CRM, ERP, payment processors, product data, spreadsheets), starting from what’s already in Databricks. Meet freshness targets and expose freshness with each fact. Track dependencies so facts and certified tables are updated or flagged when a source or definition changes. Own data quality checks, observability, and cost visibility.
• Data interfaces: Build the APIs, webhooks, and MCP tools that query and filter facts deterministically, carrying the caller’s identity and permissions on every request. Log every access with the sources it used.
• Collaboration and handover: Give the AI engineer the facts, entity IDs, and access rules that knowledge and agents depend on. Work with the full-stack engineer on the owners’ review interface and with the data quality lead on acceptance criteria. Be the technical voice on data feasibility in discovery and design sessions, and document and hand over your work to the client’s team.
• AI-augmented engineering: Use AI tools daily for SQL and Python, schema mapping, definition drafting, test generation, and documentation. Review every AI-assisted output with care: a force multiplier, not a shortcut.

Benefits

• 💵 Monthly USD Service Fee
• 🌴 20 business days of paid Agreed Time Off—Annual + 6 Local Holidays
• 🐣 Caregiver Time Off
• 💻 A dynamic remote work culture. You can work from anywhere!
• 🚀 An entrepreneurial environment.