[Hiring] Senior Software Engineer @NielsenIQ

Role Description

Design and build distributed scraping architectures that can scale across thousands of domains.

• Reverse engineer APIs and mobile apps using tools like Frida, mitmproxy, Charles Proxy, burp, and many more.
• Detect common anti-bot measures (like JavaScript checks or browser behavior detection) to build crawlers that adjust to appear more human-like.
• Lead technical innovation in scraping: identify edge cases, prototype new approaches, and share knowledge with the team.
• Set up anomaly detection and monitoring to detect blockers or failures early.

Qualifications

• 5+ years of experience with Python and large-scale scraping projects.
• Deep understanding of anti-bot technologies and how to bypass them, fingerprinting, WAFs, JavaScript challenges, etc.
• Experience working with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless browser drivers.
• Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection, and TLS tweaking.
• Strong debugging skills across browser, HTTP traffic, and device-level interactions.
• You think independently and enjoy solving problems no one has solved before.
• Familiarity with structured storage formats (Parquet, Avro), or real-time stream processing.
• Good knowledge of databases like MongoDB, InfluxDB, and Redis, including how they work and how to optimize them.

Requirements

• Experience with real-device simulation for mobile scraping.
• Comfortable building and maintaining distributed task queues (Celery, Kafka, etc.).
• Experience scraping high-security platforms or mobile-first environments.
• Exposure to Linux or cloud environments (AWS, GCP, etc.).
• Basic understanding of Kubernetes, Cloud Run, and Cloud Functions to deploy and orchestrate scraping workloads.

Benefits

• Flexible working environment.
• Volunteer time off.
• LinkedIn Learning.
• Employee-Assistance-Program (EAP).