[Hiring] Data Annotation Specialist - Computer Use Agents (CUA) Trajectory Evaluator @Careerflow.ai

Role Description

We are looking for skilled professionals to contribute as S2 Annotators, responsible for producing and validating high-quality Computer-Use Agent (CUA) trajectories for developer-adjacent workflows. This includes tasks such as file operations, light scripting, API interactions, and browser automation. This role requires a strong understanding of technical workflows, attention to detail, and the ability to translate natural language instructions into precise, step-by-step executable actions that can be used to train advanced AI systems.

What does day-to-day look like:

• Create detailed, step-by-step positive CUA trajectories for technical tasks (e.g., file manipulation, scripting, API calls, browser-based workflows)
• Break down natural language instructions into clear, verifiable actions
• Validate and review trajectories for correctness, completeness, and reproducibility
• Work within Linux desktop environments to execute and document workflows
• Use scripting (Python/Bash) to simulate or validate task execution where required
• Interact with tools and environments involving APIs, terminals, and browser automation
• Collaborate with internal teams to refine task quality and annotation guidelines
• Ensure consistency, accuracy, and high-quality standards across all annotations

Qualifications

• 2–5 years of experience in software development, technical support, or similar technical roles
• Strong familiarity with Linux environments and command-line operations
• Proficiency in at least one scripting language: Python or Bash
• Ability to decompose complex instructions into structured, step-by-step workflows
• Strong attention to detail in documenting technical processes
• Exposure to LLM-based tools, AI systems, or agentic workflows
• Basic understanding of APIs, file systems, and developer tooling
• Familiarity with OpenClaw or similar environments/tools

Requirements

• Prior experience in data annotation, RLHF, or SFT labeling workflows (nice to have)
• Exposure to CI/CD pipelines, REST APIs, or terminal-based automation (nice to have)
• Experience working with browser automation tools or developer productivity tools (nice to have)
• Background in evaluating or improving AI-generated outputs (nice to have)

Benefits

• Engagement type: Contractor assignment/freelancer (no medical/paid leave)
• Duration: 5 weeks

Evaluation Process

• Resume screening
• Take home assessment (60 mins)