Web crawler
A Web crawler is an Internet bot which systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers can copy all the pages they visit for later processing by a search engine which indexes the downloaded pages so the users can search much more efficiently. Crawlers can validate hyperlinks and HTML code. They can also be used for web scraping (see also data-driven programming).
Automated agentAutomatic indexerCrawl siteFAST CrawlerRBSEScutterSearch botSearch engine robotSearch engine robotsSearch engine spiderSearch engine spidersSearch robotSite crawlerSpider botSpider operating systemSpiderableSpiderbotSpideringWeb-crawlerWeb CrawlerWeb crawlersWeb crawlingWeb scutterWeb spiderWebcrawlerWebcrawlers
Wikipage redirect
type
primaryTopic
Web crawler
A Web crawler is an Internet bot which systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web search engines and some other sites use Web crawling or spidering software to update their web content or indices of others sites' web content. Web crawlers can copy all the pages they visit for later processing by a search engine which indexes the downloaded pages so the users can search much more efficiently. Crawlers can validate hyperlinks and HTML code. They can also be used for web scraping (see also data-driven programming).
has abstract
A Web crawler is an Internet b ...... also data-driven programming).
@en
Een spider (ook wel webcrawler ...... de webserver overbelast raakt.
@nl
Ein Webcrawler (auch Spider, S ...... erholenden Aufgaben nachgehen.
@de
Robot internetowy (lub też Rob ...... ących o pozycjonowaniu strony.
@pl
Un crawler (detto anche web cr ...... ggiornati al 21 febbraio 2008:
@it
Un robot d'indexation (ou litt ...... nt pour écrire de tels robots.
@fr
Una araña web (o araña de la w ...... os para recopilar un catálogo.
@es
Web crawler, em português rast ...... rdo com um conjunto de regras.
@pt
Поиско́вый ро́бот («веб-пау́к» ...... получить доступ к содержимому.
@ru
زاحف الشبكة (بالإنجليزية: Web ...... ة الصفحات التي تلي في الزيارة.
@ar
thumbnail
Link from a Wikipage to an external page
Wikipage page ID
Wikipage revision ID
744,766,736
hypernym
comment
A Web crawler is an Internet b ...... also data-driven programming).
@en
Een spider (ook wel webcrawler ...... oor bijvoorbeeld zoekmachines.
@nl
Ein Webcrawler (auch Spider, S ...... erholenden Aufgaben nachgehen.
@de
Robot internetowy (lub też Rob ...... je,
* tworzą mirrory stron.
@pl
Un crawler (detto anche web cr ...... ggiornati al 21 febbraio 2008:
@it
Un robot d'indexation (ou litt ...... laçable par le mot collecteur.
@fr
Una araña web (o araña de la w ...... las arañas de la web tenemos:
@es
Web crawler, em português rast ...... rdo com um conjunto de regras.
@pt
Поиско́вый ро́бот («веб-пау́к» ...... ированный сайт, чтобы определи
@ru
زاحف الشبكة (بالإنجليزية: Web ...... د يكون تم تحديثه أو حتى حذفها.
@ar
label
Araña web
@es
Crawler
@it
Rastreador web
@pt
Robot d'indexation
@fr
Robot internetowy
@pl
Spider
@nl
Web crawler
@en
Webcrawler
@de
Поисковый робот
@ru
زاحف الشبكة
@ar