[Hiring] Content Quality Analyst @The Andersen Agency

Role Description

We are seeking a detail-oriented and customer-focused Content QA Analyst to ensure the quality, reliability, and effectiveness of content experiences across Microsoft 365 products, including templates, prompt templates, AI-generated content, and emerging Copilot experiences. This role sits at the intersection of content, user experience, and quality assurance. The successful candidate will validate content performance, identify defects, evaluate AI-generated outputs, and help improve customer-facing experiences across productivity applications and AI-powered solutions.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, Content Automation, or a related field.
• Experience in quality assurance, content testing, quality control, usability testing, or related disciplines.
• Experience creating test plans, test cases, validation documentation, and test reports.
• Experience evaluating digital content and customer-facing experiences.
• Experience applying structured testing methodologies and quality assurance processes.
• Knowledge of Microsoft 365 applications and productivity workflows.
• Experience scripting with Python, PowerShell, or similar technologies.
• Strong understanding of QA methodologies, defect tracking, and quality frameworks.
• Analytical problem-solving and data interpretation skills.
• Excellent written and verbal communication skills.
• Strong attention to detail and commitment to quality.

Requirements

• Hands-on experience in content testing, QA, or QC environments.
• Proven ability to create test plans, execute test cases, and document results.
• Experience communicating testing outcomes and recommendations to stakeholders.
• Familiarity with scripting languages such as Python, PowerShell, or similar technologies.
• Ability to work independently in a remote environment and manage multiple priorities.
• Customer-focused approach with a commitment to continuous improvement and quality.

Assessment Process

• Technical Assessment
  • Review of QA, content testing, and quality assurance experience.
  • Validation of test planning, test execution, and defect management capabilities.
  • Discussion of testing methodologies, quality processes, and reporting practices.
  • Evaluation of scripting and automation knowledge.
• Scenario-Based Evaluation
  • Review of content quality evaluation techniques.
  • Assessment of AI-generated content analysis and recommendation development.
  • Defect identification and root-cause analysis exercises.
  • Sample test planning and reporting activities.
• Communication Assessment
  • Ability to clearly communicate testing results and quality recommendations.
  • Demonstration of stakeholder management and documentation skills.

Performance Measurement

• Volume and quality of testing completed.
• Effectiveness and completeness of test plans and test cases.
• Quality of defect identification, documentation, and resolution tracking.
• Timely delivery against milestones and project objectives.
• Proactive communication regarding risks, blockers, and dependencies.
• Ability to synthesize testing data into actionable recommendations.
• Contributions to process improvements, automation opportunities, and quality enhancements.

Key Responsibilities

• Content Testing & Quality Assurance
  • Develop and execute test plans for templates, prompt templates, AI-generated content, and other Microsoft 365 content experiences.
  • Perform manual and automated testing across Word, Excel, PowerPoint, Create, and Copilot experiences.
  • Validate content quality, functionality, usability, accessibility, and adherence to design standards.
  • Identify, document, reproduce, and track defects through resolution.
  • Verify content performance across a variety of customer scenarios and workflows.
• AI Content Evaluation
  • Evaluate AI-generated outputs against established quality standards and acceptance criteria.
  • Assess content accuracy, relevance, formatting, usability, and overall customer experience.
  • Provide actionable recommendations to improve content and model performance.
  • Support quality evaluation efforts for emerging generative AI experiences.
• Test Development & Execution
  • Create and maintain test plans, test cases, scripts, validation frameworks, and testing documentation.
  • Develop repeatable testing approaches for new content experiences and product features.
  • Collaborate with content creators, product teams, designers, and other stakeholders to improve testing coverage and quality.
  • Identify opportunities to streamline testing through automation and scalable processes.
• Reporting & Recommendations
  • Analyze testing results and communicate findings to stakeholders.
  • Identify recurring issues, quality trends, and areas for improvement.
  • Track testing outcomes and ensure updates are reflected in documentation and production workflows.
  • Translate testing insights into clear, actionable recommendations.

Unique Selling Points

• Work on Leading AI and Productivity Experiences
• Gain Hands-On Exposure to Generative AI
• Influence Product Quality at Scale
• Collaborate Across Multiple Disciplines
• Drive Meaningful Business Impact