[Hiring] HITL Quality Analyst @Ninja Partners LLC

Role Description

The HITL Quality Analyst is responsible for evaluating AI-generated outputs against established quality standards to ensure accuracy, consistency, and compliance with organizational requirements. This role identifies, classifies, and tracks defects, analyzes quality trends, and collaborates with cross-functional stakeholders to drive continuous improvements that enhance AI performance and operational effectiveness.

• Review and evaluate AI-generated outputs using established quality standards, evaluation criteria, and defect classification methodologies.
• Identify, classify, document, and track defects to support consistent quality measurement, trend analysis, and continuous improvement initiatives.
• Maintain accurate quality records, defect logs, and resolution tracking to ensure findings are documented and monitored through completion.
• Analyze quality metrics, detect trends, and performance data to identify recurring issues, operational risks, and opportunities for improvement.
• Coordinate with Knowledge Management, AI Tuning, Governance, Operations, and other stakeholders to communicate findings and support timely issue resolution.
• Validate the effectiveness of corrective actions by monitoring quality improvements and confirming defect resolution through follow-up evaluations.
• Support the development and refinement of quality standards, evaluation methodologies, sampling strategies, and defect taxonomies.
• Prepare reports, dashboards, and performance summaries that provide actionable insights to operational and business stakeholders.
• Participate in calibration sessions to promote consistent evaluation practices and maintain scoring accuracy across reviewers.
• Identify opportunities to improve quality processes, reporting, evaluation consistency, and operational performance through continuous improvement initiatives.
• Adhere to company policies and procedures.
• Meet or exceed performance targets for related KPIs.
• Continuously improve knowledge of products, services, and processes by participating in training programs and continuous learning modules.
• Collaborate with other departments as needed.
• Maintain a positive, empathetic, and professional attitude toward customers and colleagues at all times.
• Perform other duties as assigned.

Qualifications

• Minimum of two (2) years of experience in quality assurance, quality evaluation, compliance, auditing, transaction monitoring, operational quality, or a related field.
• Experience evaluating work against defined quality standards, scorecards, policies, or operational procedures.
• Experience identifying, categorizing, documenting, and tracking defects or quality issues to support performance improvement initiatives.
• Experience conducting calibration activities or applying standardized evaluation methodologies to ensure scoring consistency is preferred.
• Experience analyzing quality metrics, trends, and operational data to identify improvement opportunities and recurring issues.
• Strong analytical and critical-thinking skills with the ability to identify patterns, evaluate root causes, and recommend improvements.
• Strong written and verbal communication skills with the ability to clearly document findings and communicate recommendations to cross-functional stakeholders.
• Strong organizational and time management skills with the ability to manage multiple priorities while maintaining evaluation accuracy and consistency.
• Ability to work independently and collaboratively in a fast-paced environment while maintaining objectivity and attention to detail.
• Commitment to quality, operational excellence, continuous improvement, and data-driven decision-making.

Benefits

• Full-time employees.
• Competitive compensation.
• Adherence to government-mandated benefits.
• Retirement Savings Program with Company Matching.
• Life Insurance.
• HMO on day 1.
• Paid time off, birthday leave.
• Bonus and incentive plans.
• Opportunities for skills training and personal and professional development.
• Employee Referral Program.
• Beautiful office space.
• Free lunch provided daily.