[Hiring] Lead Observability Engineer @EverOps

Role Description

EverOps is looking for a Lead Observability Engineer with deep hands-on experience across metrics, logs, and traces at very large scale to lead an observability maturity assessment, and the platform consolidation that follows it, for a high-scale consumer mobile platform.

• The current estate spans multiple commercial observability vendors alongside self-managed Prometheus, Thanos, Grafana, Vector, and ELK.
• It carries tens of millions of active time series, tens of thousands of scrape targets, well over a thousand dashboards, and more than ten thousand alert definitions.
• Much of the log and trace data is sampled or dropped for cost reasons, ownership tagging is sparse, and the self-managed components carry an operational load that crowds out improvement.
• The direction under evaluation is consolidation onto AWS-native observability services built on OpenTelemetry.

This role requires someone who knows these systems well enough to price that move honestly, prove or disprove performance parity, and then lead the migration if the answer is go.

As a Lead Observability Engineer, you will join our U.S.-Based Virtual Operating Center and lead an embedded TechPod as its Pod Leader : a player-coach who sets technical direction, serves as the primary point of contact for the customer’s engineering leadership, and stays hands-on in the work.

Your immediate priority is leading a two-month Observability Maturity Assessment . You’ll validate telemetry volumes, retention, sampling, and cardinality; build a total cost of ownership model; design an AWS-native target architecture; and deliver a migration plan and commercial recommendation that leadership can make a go/no-go decision on.

As discovery closes, your focus shifts to leading the platform migration : standing up the target collection pipeline and storage tiers, porting log transforms to OpenTelemetry, rebuilding dashboards and alerts, and establishing policy-driven retention and ownership tagging that make observability both less expensive and more useful.

The customer’s observability team is capable but stretched thin. You will be expected to add capacity rather than consume it: pull the data yourself, ask targeted questions, keep recommendations unbiased, and leave the team with fewer systems to run and better telemetry than they have today.

Qualifications

• 8+ years in SRE, DevOps, Observability, or Platform Engineering.
• 4+ years owning production observability platforms.
• Prior experience in a technical lead, staff, or principal-level role.

Requirements

• Deep experience running metrics, logging, and tracing platforms at large scale (millions of active series, multiple terabytes of logs per day).
• Advanced production experience with Prometheus and a long-term storage layer such as Thanos, Cortex, or Mimir.
• Hands-on experience with Datadog or a comparable commercial platform.
• Production experience with CloudWatch, CloudWatch Logs, Container Insights, X-Ray or Application Signals.
• Production experience with the OpenTelemetry Collector (or ADOT).
• Experience designing and operating log pipelines with Vector, Fluent Bit, Logstash, or Firehose.
• Strong production experience with EKS or Kubernetes.
• Advanced proficiency with Terraform.
• Experience designing SLO-based alerting and integrating with incident management tooling.
• Ability to build defensible TCO models from usage data and pricing.
• Strong scripting ability using Python, Go, or Bash.
• Demonstrated ability to enter an unfamiliar environment and produce a defensible current-state picture and recommendation.
• Ability to explain technical, cost, and compliance tradeoffs to various stakeholders.

Benefits

• 100% Remote Workplace: We’ve been remote since Day 1!
• Unlimited Paid Time Off.
• Equity: Become a true owner of the company.
• 401K with company contribution and sponsored healthcare.
• Access to training and certification programs to accelerate your career.