[Hiring] Senior AI/ML Quality Assurance Engineer @CDM Smith

Role Description

Trinnex, wholly owned subsidiary of CDM Smith, is seeking a highly skilled and collaborative Senior AI/ML Quality Assurance Engineer to lead the validation, testing, and monitoring of production AI and machine learning solutions. This role is responsible for ensuring our predictive models are accurate, reliable, explainable, and compliant, while partnering with Data Science, MLOps, Consulting, and Governance teams to support responsible AI delivery.

The ideal candidate combines expertise in AI/ML testing, model performance monitoring, and automation with the ability to translate complex technical findings into clear, actionable insights for both technical and non-technical stakeholders. This position will play a critical role in maintaining the quality and trustworthiness of Trinnex's AI-powered products across infrastructure, environmental compliance, and operational monitoring solutions.

Key Responsibilities

• Design, develop, and maintain automated testing frameworks to validate machine learning models, data pipelines, and predictive analytics solutions.
• Validate tabular, geospatial, classification, and risk-scoring models to ensure accuracy, reliability, and alignment with business and regulatory requirements.
• Perform model explainability and quality assessments using industry-standard XAI techniques and document results for internal and client-facing stakeholders.
• Partner with MLOps teams to implement monitoring, alerting, and performance tracking for data drift, concept drift, and model degradation in production environments.
• Support root cause analysis, incident investigations, rollback validation, and recovery testing for AI/ML systems.
• Assist in evaluating emerging Generative AI and Retrieval-Augmented Generation (RAG) solutions, including model performance, retrieval accuracy, safety, and guardrail effectiveness.
• Collaborate with Consulting and Governance teams to develop validation reports, model documentation, and compliance artifacts that support transparent and responsible AI practices.
• Conduct quality reviews of AI and data science deliverables prior to client release, identifying risks, data quality concerns, and model limitations.

Qualifications

• Bachelor's degree.
• 5 years of related experience.
• Equivalent additional directly related experience will be considered in lieu of a college degree.

Preferred Qualifications

• Bachelor's degree in Computer Science, Software Engineering, Data Science, Geographic Information Systems (GIS), Engineering, or a related quantitative field.
• Experience validating machine learning models, datasets, or analytical solutions within infrastructure, utilities, environmental compliance, water, or wastewater domains.
• Familiarity with geospatial data, spatial analytics, or GIS-based applications.
• Experience working with time-series, sensor, IoT, or SCADA data in operational environments.
• Knowledge of AI governance, model risk management, or responsible AI frameworks.
• Familiarity with cloud-based AI/ML platforms and production monitoring environments.
• Experience evaluating or testing Generative AI, large language model (LLM), or Retrieval-Augmented Generation (RAG) solutions.
• Understanding model explainability, fairness, bias assessment, and AI transparency concepts.

Skills & Abilities

• Experience validating, testing, or monitoring production machine learning models, particularly classification, regression, or predictive analytics solutions.
• Strong proficiency in Python, SQL, and Git, with hands-on experience handling spatial data structures (e.g., GeoPandas, Shapely, or PostGIS).
• Experience with AI/ML testing, validation, and monitoring frameworks, including automated testing, model performance monitoring, and observability solutions (e.g. pytest, MLflow, or Deepchecks; Evidently AI, Arize, or cloud observability stacks like GCP Production Monitoring/Cloud Monitoring, Grafana, or Datadog).
• Familiarity with Docker, Kubernetes, and cloud environments (GCP), including application monitoring and production support workflows.
• Ability to translate complex data science and model outputs into clear, actionable documentation for technical and business stakeholders.
• Understanding of model performance measurement, drift detection, and production monitoring concepts.
• Strong technical writing skills with experience creating and reviewing model documentation, validation reports, test plans, runbooks, APIs, or other technical artifacts.
• Ability to collaborate across multidisciplinary teams, including Data Science, MLOps, Software Engineering, Product Management, and Consulting organizations.

Amount of Travel Required

• No Travel is required.

Pay Range

• Min: CAD $144,477.00
• Max: CAD $252,824.00

Additional Information

• This job posting is for an existing vacancy.
• Background checks and drug testing may be required.
• Visa Sponsorship Available: No.