Member of Technical Staff (Site Reliability Engineer, Crawler)

Perplexity
London, United Kingdom

Who this role is best for

Geared toward infrastructure-focused engineers comfortable with distributed systems optimization, with a preference for candidates with experience in crawler or search infrastructure domains and a willingness to engage in on-call rotations.

Best fit for

  • Candidates with experience in scaling distributed systems and defining SLOs for reliability improvements
    — “Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation.”
  • Individuals who have led incident response and implemented durable fixes in production environments
    — “Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements.”
  • Engineers with strong hands-on skills in Kubernetes and cloud infrastructure management
    — “Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services.”

Things to consider

  • On-call rotation participation is expected, which may require availability outside standard hours
    — “Participate in the team's on-call rotation, lead incident response”
  • Candidates should be prepared to diagnose complex distributed system failures regularly
    — “Proven experience diagnosing distributed-system failures”

How to stand out

  • Highlight experience with capacity planning and resource efficiency in distributed environments
    — “Own capacity planning and resource efficiency; diagnose queue backlogs”
  • Showcase skills in building CI/CD pipelines with validation and rollback capabilities
    — “Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback.”
  • Emphasize your track record in leading reliability improvements through postmortems
    — “turn blameless postmortems into durable reliability improvements.”
  • Demonstrate your ability to write production software for automated diagnosis and remediation
    — “Write production software to automate diagnosis and routine remediation”
Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • improve throughput
  • plan capacity
  • build operational reliability
  • diagnose distributed-system failures
Typical background
5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles

Skills & requirements

Required

SREProduction EngineeringKubernetesInfrastructure As CodeCi/cd PipelinesObservabilityLinuxNetworking

Preferred

Web CrawlersSearch InfrastructureBrowser FleetsDistributed QueuesBackpressureWorkload IsolationMulti-region Recovery

Stack & domain

PythonC++RustGoKubernetesTerraformLinuxNetworkingTcp/ipDNSTLSHTTPLoad BalancingDatadogPrometheusGrafanaSlosDiagnosisIncident ResponseBlameless PostmortemsCloud InfrastructureProduction EngineeringBackend RolesObservabilityReliability

About the role

Original posting from Perplexity via Ashby

Keeping a crawler running is not the same as keeping it productive. Queues can grow while servers look healthy, and a small change can become a fleet-wide bottleneck. On Perplexity's Crawler team, you'll improve throughput, plan capacity, and build operational reliability through safer deployments, better observability, and durable fixes.

Responsibilities

  • Operate and optimize Kubernetes workloads and supporting infrastructure for crawler services, including fetching and rendering fleets.
  • Build and maintain infrastructure as code and CI/CD pipelines, with safe rollouts, validation, and fast rollback.
  • Define SLOs and build monitoring, alerting, and observability for availability, latency, throughput, and resource saturation.
  • Own capacity planning and resource efficiency; diagnose queue backlogs, network failures, and resource bottlenecks with service owners.
  • Participate in the team's on-call rotation, lead incident response, and turn blameless postmortems into durable reliability improvements.
  • Write production software to automate diagnosis and routine remediation, with validation and approval gates for high-impact actions.

Qualifications

  • 5+ years of experience in SRE, production engineering, or infrastructure-heavy backend roles, with strong programming skills in Python, C++, Rust, or Go.
  • Hands-on experience with cloud infrastructure, production Kubernetes, infrastructure as code such as Terraform, and automated deployment workflows.
  • Strong Linux and networking fundamentals, including TCP/IP, DNS, TLS, HTTP, and load balancing.
  • Experience building observability with tools such as Datadog, Prometheus, or Grafana, and using SLOs to guide reliability improvements.
  • Proven experience diagnosing distributed-system failures, leading production incidents, and improving system performance or resource efficiency.

Nice to have

  • Experience operating web crawlers, search infrastructure, or browser fleets at scale.
  • Experience with distributed queues, backpressure, workload isolation, or multi-region recovery.

Source: Perplexity careers (Ashby)

Similar roles

More roles at Perplexity