[Hiring] Staff Software Engineer, OTA Services @Horizon3

Role Description

The Expert Engineer on the OTA Services team is the technical anchor for how NodeZero updates itself across thousands of customer deployments — without customer intervention, without downtime, and without introducing new vulnerabilities in the process. This is a hands-on staff-level role.

• Write production code, produce architecture designs, set the technical direction for update delivery and version management, and mentor P4 engineers on the team.
• Work in close partnership with the Image Build & Release team.
• Report to the Client Platform Engineering Manager.

Responsibilities

• Design and own the over-the-air update system — the delivery protocol, version negotiation, rollout orchestration, health verification, and automatic rollback.
• Own the systems that track the version state of every NodeZero deployment — know what's running where, what's pending, and what's stuck.
• Design safe rollout strategies for updates — phased rollouts, canary deployments, customer pinning, and emergency hold mechanisms.
• Design delivery mechanisms that work reliably across diverse customer environments including air-gapped networks, strict egress policies, and minimal-footprint deployments.
• Own OTA system reliability end-to-end — on-call for delivery failures, stuck updates, and version drift incidents.
• Mentor P4 engineers through design review, code review, and technical pairing.

Qualifications

• 8+ years of professional software engineering experience with deep specialization in distributed systems, remote update systems, or fleet management infrastructure.
• Deep understanding of remote update system design — delivery protocols, version negotiation, delta delivery, health verification, and rollback semantics.
• Experience designing rollout strategies for software deployed across heterogeneous, customer-managed environments.
• Strong proficiency in Go, Python, or Rust for systems-level infrastructure work.
• Experience building software that operates in constrained or air-gapped environments.
• Deep understanding of Kubernetes — how deployments are managed, updated, and monitored in running clusters.
• Experience with container image distribution: OCI artifacts, image manifests, layer caching, and registry protocol.
• Strong understanding of mTLS, certificate lifecycle, and secure channel establishment for update delivery.
• Track record of setting engineering standards that other engineers adopt.
• Ability to produce clear architecture documents and drive technical decisions across team boundaries.
• Experience mentoring mid-level engineers through design review and code review.

Nice to Haves

• Experience with Kubernetes operators or controller patterns.
• Background in cybersecurity or experience building software for regulated or high-security environments.
• Familiarity with Fleet (or similar open-source fleet management tools).
• Experience with signed artifact delivery and chain-of-custody verification end-to-end.

Travel Required

We are a fully remote company, and this job may require up to 5% of travel to be successful.

Compensation and Values

At Horizon3, we believe that our people are our greatest asset, and our compensation philosophy reflects this core value.

• Base salary range: $229,500 - $260,000 annually.
• Additional compensation may include an equity package (in the form of stock options).

Benefits

• Inclusive Team: We value diversity and promote an inclusive culture where everyone can thrive.
• Growth Opportunities: Be part of a dynamic and growing team with numerous career development opportunities.
• Innovative Culture: Work in a collaborative environment that encourages creativity and out-of-the-box thinking.
• Remote Work: We are a 100% remote company.
• Competitive Compensation: Includes health, vision & dental care for you and your family, a flexible vacation policy, and generous parental leave.

You Belong Here

Horizon3 is not just an equal opportunity employer - we are a community that values diversity, equity, and inclusion as fundamental principles of our culture and success.

Other Duties

Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee.

Application Note

In any materials you submit, you may redact or remove age-identifying information.