If you do not find the required information using the tips from the previous section, then you need to consult an online service to find the piece of information you were looking for. Three different types of services have been established over the years: crawlers, directories and meta search engines. The first crawler was created back in 1993. It was called the World Wide Worm. It crawled from one site to the next and indexed all pages by saving the content of the web pages into a huge database. Crawlers or spiders visit a web page, read it, and then follow links to other pages within the site and even follow links to other sites. Web crawlers return to each site on a regular basis, such as every month or two, to look for changes. Everything a crawler finds goes into a database, which people are able to query. The advantage of the web crawlers is that they have an extensive database with almost the complete Internet indexed in it. The disadvantage is that you get thousands of web pages as a response for almost any request. Web directories work a little differently. First of all they contain a structured tree of information. All information entered into this tree is either entered by the webmaster who wants to announce his new web page or by the directory maintainer who looks at the web pages submitted. The directories where webmasters can submit both the URL (Uniform Resource Locator) and the description normally contain more misleading information. Getting into most web directories is a combination of luck and quality. Although anyone is able to submit a web page, there is no guarantee that a certain page will be included. Some directories charge for submission, which is basically a rip-off. Some crawlers maintain an associated directory. If your site is in the directory, the ranking of your web site normally will be higher, as it is already pre-classified. Many directories work with crawlers together, in order to deliver results, when nothing can be found in the directory. Meta search engines do not have a database with URLs and descriptions. Instead they have a database of search engines. If you enter your keywords into a meta search engine it will send out requests to all the directories and crawlers it has stored in its database. The meta search engines with a more sophisticated application in the background are able to detect double URLs which come back from the various search engines and present only a single URL to the customer.
34. Crawler is a kind of__________.
A. worm B. search engine C. service D. directory
35. The disadvantage of Crawler is that it__________.
A. provides a complete database
B. takes too much time to find the needed information
C. provides too much useless information
D. gives response to every quest
36. Who can input information to the web directories?
A. Only webmasters.
B. Both directory maintainers and webmasters.
C. Any one who wants to announce a web page.
D. Any one who has found a new web page.
