[Hiring] Data Engineer @Public Sector Resourcing

Role Description

On behalf of National Crime Agency, we are looking for a Data Engineer Inside IR35 for a 6 month contract based REMOTELY.

As a Data Engineer, your main responsibilities will be:

• Design, build and maintain scalable data pipelines within Microsoft Azure
• Migrate and transform data from legacy Oracle systems into cloud-based platforms
• Develop and support ETL/ELT processes using Azure Data Factory and Azure Databricks
• Perform data mapping, schema transformation and data modelling activities
• Manage and optimise data movement into Azure Data Lake environments
• Work with Delta Lake/Delta formats to support efficient data storage and processing
• Ensure high levels of data quality, integrity and reliability across all datasets
• Troubleshoot and resolve data-related issues and pipeline failures
• Support both batch and real-time (streaming) data processing requirements
• Produce clear technical documentation, including data dictionaries and metadata records
• Contribute to data governance activities including data lineage, classification and quality controls
• Design secure, scalable and high-performing data solutions
• Work collaboratively with project teams and stakeholders to meet programme objectives
• Deliver knowledge transfer and handover sessions to ensure long-term supportability

Qualifications

• Proven experience working as a Data Engineer in large-scale data migration projects
• Strong experience migrating data from Oracle databases into Microsoft Azure
• Hands-on expertise with Azure Databricks
• Experience using Azure Data Factory (ADF)
• Experience working with Azure Data Lake technologies
• Knowledge of Delta Lake / Delta formats
• Strong ETL/ELT development experience
• Data mapping and schema transformation expertise
• Excellent SQL and data integration skills
• Understanding of data governance principles, including:
• Data lineage
• Data classification
• Data quality management
• Metadata management
• Experience creating and maintaining technical documentation and data dictionaries
• Ability to build secure, scalable and performant data solutions
• Strong stakeholder communication skills

Requirements

• Active SC Clearance
• Active SC/DV Clearance (Please select level of clearance required)
• Please be aware that this role can only be worked within the UK and not Overseas.

Desirable Experience

• Azure Synapse Analytics
• Microsoft Fabric
• Python and/or PySpark
• Spark-based data processing
• Experience within government, policing, defence or other highly regulated environments

Benefits

• Disability Confident: As a Disability Confident employer, CA encourage applications from disabled and neurodivergent candidates.
• Applications from candidates who identify as having a disability and meet the essential criteria for the role will be prioritised for review.

Company Description

In applying for this role, you acknowledge the following: "this role falls in scope of the Off Payroll Working in the Public Sector legislation. Any rates of payment quoted will reflect the gross rate per day for the assignment and will be subject to appropriate taxes and statutory costs. As such the payment to the intermediary and your income resulting from this contract will be different".