Web Crawler

Enables web crawling and scraping functionality with configurable parameters like crawling depth, request delay, and concurrency. Allows fetching content from websites with customizable timeout and link following settings.