[Hiring] Staff Engineer - Distributed Systems @HighLevel

Role Description

Your job is to find the failure before it finds production. You start with Workflows, HighLevel's automation engine, and one of the largest systems in the company:

• 3.1 billion enrollments
• 21.5 billion action executions every month
• Traffic peaking at 28,000+ requests per second
• Running across thousands of pods on GCP with Pub/Sub, Cloud Tasks, Redis, and multiple database engines underneath

From there, your blast radius grows into Conversations (2.6 billion messages a month) and a brand-new Ticketing system being built right now, where you get to make sure it's born right instead of fixed later.

This is not an architect role where you draw boxes and hand them to someone else. And it's not a feature role where you own a backlog. It's the role in between that most companies never create, and most staff engineers spend their careers wishing existed.

Responsibilities

• Own the architecture health of a billion-scale distributed system, its failure modes, capacity limits, consistency guarantees, and the interactions between 50+ deployments that no single team can see.
• Approve critical-path designs. Changes that touch the system's core go through you, not as bureaucracy, but as the person accountable for the whole staying sound.
• Hunt gaps proactively, single points of failure, unbounded queues, missing idempotency, thundering herds, quiet data-loss windows, and drive the fixes before they become incidents.
• Build the parts nobody else can. No sprint tickets. You prototype the risky architectural bets yourself, ship the remediation after serious incidents, and pair into the gnarliest cross-team bugs.
• Make resilience a property of the system, not a heroic act, degradation strategies, backpressure, isolation boundaries, capacity models that survive 10%+ month-over-month growth.
• Raise the teams around you. Design reviews that teach, post-mortems that change architecture (not just add alerts), and patterns that 80+ engineers build on.
• Set the standard for how AI-assisted engineering works safely on systems this critical.

Qualifications

• 10+ years of engineering experience with deep, hands-on ownership of large-scale distributed systems.
• You've carried sole accountability for a production system through real failures.
• Deep command of queueing and async architectures, delivery semantics, ordering, backpressure, idempotency.
• Strong with multiple storage engines (SQL and NoSQL).
• Expert-level depth in Redis or comparable in-memory systems.
• Production experience on Kubernetes at scale.
• Exceptional design communication, docs, diagrams, and RCAs that drive decisions across multiple teams.
• Fluent in Node.js and/or Go.

Requirements

• The critical paths of Workflows have named owners, capacity models, and tested failure modes.
• Incident count trends down while traffic grows double-digit percent month over month.
• Your design reviews are the ones engineers want their proposals to survive.
• Your scope has expanded on results, more systems, more surface, more trust.

Bonus Points

• You've made AI agents genuinely productive on complex systems.
• GCP-native experience: Pub/Sub, Cloud Tasks, GKE, Firestore.
• You've done this job before under another name, "the systems person," principal engineer, architect-who-still-codes.
• Experience taking a 0→1 system to production alongside hardening mature ones.