Scraping Project Manager

About the role

Livetrend is a B2B SaaS company in the fashion-tech space that transforms large volumes of market data into actionable insights. The company is looking for a Scraping Project Manager to own and operate its daily scraping operations, which run around 400 web scrapers. This senior-hands-on role focuses on technical decision-making, quality assurance and code review, and is fully remote on a permanent contract.

Key responsibilities

• Own scraping projects end-to-end, from specification through to deliverable validation.
• Make technical choices for each target site, estimating cost and performance trade-offs.
• Challenge and verify deliverables, reviewing code and raising the quality of shipped scrapers.
• Review and mentor junior engineers’ code.
• Manage and maintain a mixed proxy pool (datacenter, residential, mobile, unlocker) tailored to site constraints.
• Size scraper throughput, track and optimise cost per site.
• Resolve complex challenges such as CDN bypass, aggressive anti-bot protection, closed catalog APIs and large catalog handling.

Required profile

• Approximately 2 years of hands-on experience running web-scraping pipelines in production.
• Proven experience managing a proxy mix (datacenter, residential, mobile, unlocker).
• Experience sizing throughput and monitoring cost-per-site.
• At least one year dealing with complex scraping obstacles (CDNs, anti-bot, closed APIs, large catalogs).
• Ability to specify requirements, challenge deliverables and write or debug Python code.
• Comfortable reviewing others’ code and providing actionable feedback.
• Professional working proficiency in English.

Required skills

• Python
• Scrapy
• Playwright
• Camoufox