Your question is Web Crawler Design. Take a moment with it on the right.
Talk me through your thinking if you like. When you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes).
Imagine that you are designing a web crawler. This web crawler accepts links from users and then crawls the domain for the link and notifies the users when the crawl is done. The crawler should save the web pages and find other links on the web pages as it crawls.
Asked in the system design 2 stage. Reported follow-ups: failure scenarios for the messaging queue. Focus on how you would make crawling reliable, scalable, and observable.