[Hiring] Python/Spark/AI Developer @MSP Staffing Pty

Role Description

Our client is seeking an experienced Python/Spark/AI Developer to join a technically driven team focused on modernising data platforms and building scalable AI-enabled solutions. This role is ideal for a developer who enjoys working with distributed data, clean and testable Python, APIs, and modern lakehouse technologies.

Qualifications

• Bachelor's degree in Computer Science, Engineering or equivalent experience.
• 4+ years' professional Python development experience.
• 2+ years' production experience developing Spark or comparable distributed data pipelines.

Requirements

• Strong Python 3.12 skills, including type hints, Pydantic and modern packaging practices.
• Solid Apache Spark/PySpark knowledge, including Spark SQL, DataFrame APIs, partitioning and performance optimisation.
• Experience with Delta Lake or equivalent technologies such as Iceberg or Hudi.
• Strong SQL skills and the ability to translate legacy T-SQL logic into Spark SQL.
• Practical experience with pytest and automated testing.
• Experience developing REST APIs with FastAPI or equivalent technologies.
• Familiarity with Docker-based development environments.
• Strong Git and CI/CD practices.

Benefits

• Experience integrating LLMs and AI services.
• Knowledge of data privacy, PII handling, POPIA/GDPR concepts and data governance.
• Exposure to Hive Metastore, Trino or Apache Ranger.
• Azure data platform experience, including ADLS Gen2, Synapse or ADF.
• Experience with observability tools and structured logging.
• Previous data migration and pipeline parity experience.
• Databricks or equivalent Spark certification.

Application Instructions

Should you meet the requirements for this position, please email your CV to [email protected] . You can also contact the IT team on 031 350 3607 or visit our website at https://mspstaffingza.co/ .

NOTE: When replying to the advert, also include the reference number in the subject line. Correspondence will only be conducted with short listed candidates. Should you not hear from us within 3 days, please consider your application unsuccessful.