[Hiring] Lead Pyspark Developer @Luxoft

Role Description

The Surveillance Enhancement Program (SEP) is a multi-year program aimed at building and enhancing the surveillance platform to detect potential instances of market misconduct, market manipulation, and market abuse. This project will focus on:

• Alerts enhancements
• Strategic data sourcing from FM order and trade lakes
• UI/workflow enhancements to meet regulatory obligations (i.e. Market Abuse Regulation, Dodd Frank)
• Control Room & Suitability Monitoring related data, UI, and scenario enhancements

Responsibilities include:

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed
• Perform code reviews and recommend improvements
• Review business requirements and recommend changes to develop reliable applications
• Develop coding documentation and other technical specifications for assigned projects
• Act as primary contact for development queries and concerns
• Analyze and resolve development issues accurately

Qualifications

• 15+ years of experience in data intensive Pyspark development, Window Functions, Performance Optimization, Spark Architecture, Salting, Column Pruning
• Experience in leading a Development Team
• Proven experience as a Core Python developer
• Good understanding of Gen AI
• Experience developing Classes, OOPS, exception handling, parallel processing
• Strong knowledge of DB connectivity, data loading, transformation, calculation
• Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
• Lambda Functions, Decorators
• Vector operations on Pandas dataframes/series
• Application of apply map, apply, map functions
• Concurrency and error handling data pipeline batch of size [1-10 gb]
• Ability to understand business requirements and translate them into technical requirements
• Ability to design architecture of data pipeline for concurrent data processing
• Working with Git source control
• Service-oriented architecture, including the ability to consider integrations with other applications and services
• Debugging application

Requirements

• Knowledge of web backend technology (Django, Python, PostgreSQL)
• Apache Airflow
• Atlassian Jira
• Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
• Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus

Other

• Languages: English: C2 Proficient
• Seniority: Lead