Guys i am building a simple web crawler which crawls a website and fetches for url’s,Displays the url’s and after completely crawling a url crawls all the url’s found for a period of 10mins.
I am using regular expressions to get the links after getting the source code.
Please suggest better ideas. thnx
Check out crawler developed by me on
For educational purpose only
1 Like