Lista roboti – cautare internet – web crawlers

Search engine robots and others

The following table lists the search engines that spider the web, the IP addresses that they use, and the robot names they send out to visit your site. Version numbers are usually included in the robot names, but are omitted here except where it implies a visit from a different IP address or (as in inktomi) a different search engine.

Often multiple IP addresses are used, in which case we just give a flavour of the names or numbers. Inktomi is a company that offers search engine technology and is used by a number of sites (e.g. and

Wherever appears this indicates a number of different digits may be used.

Home page/search engine Robot identifier AbachoBOT abcdatos_botlink AESOP_com_SpiderMan crawler ( ia_archiver Scooter Mercator Scooter2_Mercator_3-1.0 Tv_Merc_resh_26_1_D-1.0 AltaVista-Intranet FAST-WebCrawler
Wget Acoon Robot antibot Atomz AxmoRobot Buscaplus Robi CanSeek/ ChristCRAWLER Clushbot Crawler ROBOT/ RaBot Agent-admin/ contact/
RaBot Agent-admin/ DeepIndex DittoSpyder Jack Speedy Spider ArchitextSpider
(excite) ArchitectSpider EuripBot Arachnoidea EZResult Fast PartnerSite Crawler FAST Data Search Crawler FAST Data Search Document Retriever KIT-Fireball FyberSearch GalaxyBot geckobot (Genealogical Search Engine) GenCrawler GeonaBot getRAX Googlebot moget/2.0 Aranha
(inktomi) Slurp/2.0j
(inktomi) Slurp/2.0-KiteHourly;
(inktomi) Slurp/2.0-OwlWeekly
(inktomi) Slurp/3.0-AU (need V5 browsers to view) Toutatis 2.5-2 Hubater (research centre) IlTrovatore-Setaccio IncyWincy UltraSeek InfoSeek Sidewinder Mole2/1.0 MP3Bot kuloko-bot/0.2 LNSpiderguy Linknzbot lookbot MantraAgent (see also NetResearchServer Lycos_Spider_(T-Rex) JoocerBot HenryTheMiragoRobot MojeekBot mozDex/ MSNBOT/0.1 Navadoo Crawler Gulliver ObjectsSearch/0.01 OnetSzukaj/ PicoSearch/ PJspider but it won’t let us in :-( DIIbot nttdirectory_robot griffon Spider/
??? gazz/1.0 NationalDirectory-SuperSpider dloader(NaverRobot)/ dumrobo(NaverRobot)/ noxtrumbot/ (Chinese language) Openfind piranha,Shark Openbot/ psbot CrawlerBoy QweeryBot AlkalineBOT StackRambler/ SeznamBot Search-10 Fluffy the spider Scrubby/ asterias speedfind ramBot xtreme Kototoi/0.1 SearchByUsa Searchspider/ SightQuestBot/ Spider_Monkey/ Surfnomore Spider v1.1 Robot@SuperSnooper.Com teoma_agent1 Teradex_Mapper ESISmartSpider Spider TraficDublu Tutorial Crawler updated/0.1beta UK Searcher Spider (coming soon) Vivante Link Checker appie Nazilla marvin/infoseek MuscatFerret WhizBang! Lab ZyBorg ( WIRE WebRefiner: WSCbot Yandex pet-based search engine Yellopet-Spider Findexa Crawler YBSbot search engine indexer


Most browsers identify themselves with a string that begins “Mozilla…”. I’ve chosen not to document those (as yet). Here are a few of the rarer browser identifiers that I’ve seen.

Browser identifier Information
AmigaVoyager Voyager browser for the Amiga
xChaos_Arachne (DOS-compatible browser. Linux version under development)
IBrowse (search for IBrowse) Amiga-based browser
ICab (Macintosh-only)
JustView (I think this is a browser. Site is in Japanese)
KMeleon (Light browser based on the Mozilla code base)
Konqueror (Linux KDE browser)
Lynx (Cross-platform text based browser)
OmniWeb (Macintosh-only)
Opera (Cross-platform, small, efficient and standards lead browser)
Plucker (Palm handhelds. Written in Python)
pwWebSpeak Audio Browser
QWeb (Linux browser) (see also
retawq Text-based browser for text terminals. Runs under Linux
SlimBrowser Freeware tabbed browser
Sleipnir (Japanese) Japanese browser with apparantly an English version available.
VMS_Mosaic (OpenVMS only version of Mosaic, a pre-Netscape browser)
WannaBe (Macintosh text-only browser)
w3m (text-based browser)
Tags: ,

About magazinweb

filme subtitrate - filme romanesti - desene animate - diverse - internet.

0 comentarii

Leave a Reply

you may leave here a comment .
thank you !