Posts tagged crawling
Q&A: FROM WHERE CAN I GET A WEB DATA SET TO BUILD SEARCH ENGINE WITHOUT CRAWLING?
Oct 29th
Question by ahmed b:
from where can i get a web data set to build search engine without crawling?
i want to build an expermental search engnie and i want web data set for a collection of crawled
web site , so from where i can get it ?
Read More >>
SEARCH ENGINE && CRAWLING ROBOTS && PROGRAMMING?
Oct 1st
Ho do robots like google robot crawl websites and index them all, I have a project that could really use a similar bot to crawl websites and retrieve infro from them and database the link of the site.
Please point me in the right direction of what I would need to do to build one? and level of difficulty?
PS: I know python, javascript, ajax, and php.
Thanks so much appreciate it!
UPDATE: Yeah I only plan to crawl a choose number of sites.. not everything in the world!
maybe 10 or so.. but appreciate the warning of search engine site, I am not going to compete with google tho!!! 
Read More >>
FROM WHERE CAN I GET A WEB DATA SET TO BUILD SEARCH ENGINE WITHOUT CRAWLING?
Jun 25th
i want to build an expermental search engnie and i want web data set for a collection of crawled
web site , so from where i can get it ?
Read More >>