[Hiring] Senior Software Engineer, ML Platform @Stack AV

Role Description

In the ML Data Understanding team, our mission is to provide trusted and useful data to efficiently power all of Stack's ML applications end-to-end from mining to training to safety evaluation. We work hand-in-hand with AV autonomy teams to provide cutting-edge solutions to all their data needs, working across data engineering, mining, modeling and infrastructure. In particular, we provide services to find, curate, annotate, search and serve data for all ML needs.

• Data Mining: Building a framework and infrastructure to find interesting events quickly and flexibly, including setting the direction for and helping to build an inference service using LLMs, open-world models, and vector databases.
• Semantic Search for Data Mining: Developing a highly scalable semantic search service for multimodal data to find interesting events quickly and flexibly, including setting the direction for and helping to build an inference service using the latest AI models & approaches.
• Dataset Management for Training: Building state-of-the-art infrastructure to support machine learning training and inference workloads using OSS components such as Ray, Spark, Lance, and Iceberg.

Qualifications

• Experience with both ML platforms and building ML-based applications.
• 5+ years of relevant domain experience.
• Proven track record of building scalable, reliable infrastructure in a fast-paced environment.
• Deep understanding of design trade-offs with the ability to articulate those trade-offs and achieve alignment with others.
• Hands-on background in developing semantic search infrastructure, generating multimodal (video/image/audio) embeddings, and managing vector databases.
• Experience with multi-modal data indexing, model training infrastructure, model optimization, or large data processing pipelines.
• Experience in building ML models or infrastructure in autonomous vehicles, perception, and decision-making domains (desirable but not required).

Requirements

• Leverage AI-assisted software engineering to architect next-generation multimodal semantic search and data discovery platforms for autonomous driving development.
• Investigate and apply frontier large language models, visual language models, and state-of-the-art Retrieval Augmented Generation (RAG) techniques.
• Build out agentic data mining, semantic search, and data curation workflows as part of a recursive self-improvement machine learning loop.
• Develop data understanding platform infrastructure for real-time querying/vector databases and batch/stream processing using technologies like Ray, Spark, LanceDB, or similar.
• Deliver end-to-end data mining solutions that span onboard (C++) and offboard (ML & Data Infra) infrastructure to accelerate AV product development.
• Develop e2e solution for real-time semantic search services (text/images/videos) and vector DBs.
• Discover and identify key issues in existing ML infra and proactively improve system performance.
• Build low latency/high throughput batch or stream processing pipelines.
• Drive technical discussions across multiple orgs and deliver solutions on a timely basis.
• Write readable and high-performance Python/C++ code.

Benefits

• We are proud to be an equal opportunity workplace.
• We believe that diverse teams produce the best ideas and outcomes.
• We are committed to building a culture of inclusion, entrepreneurship, and innovation across gender, race, age, sexual orientation, religion, disability, and identity.