Dataford
Interview QuestionsInterview GuidesExperiencesMock InterviewsPricing
Get started

Scalable Scraping With Scrapy

Hard
PipelinesBatch ProcessingOrchestrationAsked 1 times

Problem

How would you design a scalable web scraping pipeline using Scrapy to crawl financial reports from hundreds of energy company websites daily for Rystad Energy?

You are practicing as a guest. Sign up free to get your answer graded with AI feedback. Your draft stays right here.

Sign up freeI have an account
Sign up to unlock solutions
Rystad Energy Software Engineer Interview Questions
Next questions
Rystad EnergyScraped Data Pipeline Without BottlenecksHardTrexquant InvestmentScalable Scraping Without Getting BlockedHardRystad EnergyHandling Dynamic Sites and Anti-ScrapingMedium
0 / ~200 words