| sait/Nutch-0.9 |
|
|
30-set-07 |
01-set-07 |
| sallatiBOT_sallati.comV1.0 |
|
Web development & SEO - http://www.sallati.com |
30-set-04 |
01-set-04 |
| savvybot/0.2 |
|
|
31-mar-05 |
01-mar-05 |
| SBIder/0.7 |
Sitesell COM |
|
30-set-05 |
01-set-05 |
| SBIder/0.8-dev |
Sitesell COM |
|
30-apr-06 |
01-set-05 |
| SboongBot |
Sboong.com/it |
Estrae indirizzi e-mail da pagine web e da usenet per alimentare il DB del motore di ricerca e-mail |
30-giu-04 |
01-mar-04 |
| ScanWebBot/1.0 |
|
|
30-set-06 |
01-giu-06 |
| Schmozilla/v9.14 Platinum |
|
? Fetching a URL from a Perl Script |
30-giu-05 |
01-feb-04 |
| Scooter/2.0 |
Altavista COM |
non attivo dal 07/04 |
31-lug-04 |
01-lug-04 |
| Scooter/3.3 |
Altavista COM |
|
30-set-07 |
01-gen-04 |
| Scooter/3.3.vscooter |
Altavista COM |
non attivo dal 03/04 |
31-mar-04 |
01-gen-04 |
| Scooter/3.3_SF |
Altavista COM |
non attivo dal 04/04 |
30-apr-04 |
01-gen-04 |
| Scooter/3.3Y!CrawlX |
Altavista COM |
non attivo dal 08/04 |
31-ago-04 |
01-ago-04 |
| Scorpion V1.0 Crawler |
|
|
31-ago-04 |
01-ago-04 |
| Scrubby/2.1 |
ScrubTheWeb COM |
|
29-feb-04 |
01-feb-04 |
| Scrubby/2.2 |
ScrubTheWeb COM |
|
30-set-07 |
01-feb-04 |
| Scrubby/3.0 |
ScrubTheWeb COM |
|
30-set-07 |
01-ago-07 |
| ScSpider/0.2 |
|
|
30-set-07 |
01-lug-04 |
| Scumbot/5.5 |
|
|
30-apr-06 |
01-apr-06 |
| Search Engine World Robots.txt Validator at http://www.searchengineworld.com |
|
searchengineworld.com |
31-ott-04 |
01-ott-04 |
| Search Fst |
|
|
30-giu-04 |
01-giu-04 |
| search.updated.com/0.06 |
Updated COM |
|
30-nov-04 |
01-nov-04 |
| SearchBlox |
|
|
30-set-07 |
01-ago-07 |
| Search-Channel |
SearchChannell (FRANCIA) |
Adult Search Channel |
31-gen-05 |
01-apr-04 |
| searchengineBot 1.1 |
|
? searchenginebot.com |
30-nov-04 |
01-nov-04 |
| SearchGuild DMOZ Experiment |
|
|
31-ago-05 |
01-ago-05 |
| SearchSight/2.0 |
|
|
30-giu-06 |
01-ott-05 |
| SearchSight_Crawler/2.0 |
|
|
30-set-05 |
01-set-05 |
| SearchWarp_Crawler/2.0 |
|
|
31-ago-05 |
01-ago-05 |
| SEB Spider |
|
|
31-ott-05 |
01-set-05 |
| SecretBrowser/007 |
|
|
31-ago-05 |
01-mag-04 |
| Seekbot/1.0 |
Seekbot NET |
Germania |
30-set-07 |
01-ago-04 |
| Seekbot/2.x |
|
|
30-set-05 |
01-set-05 |
| Selflinkchecker 1.0 |
|
|
31-mag-06 |
01-mag-06 |
| semanticdiscovery/0.2 |
|
Domain checking tool |
31-lug-04 |
01-gen-04 |
| semanticdiscovery/0.3 |
|
Domain checking tool |
30-nov-04 |
01-lug-04 |
| semanticdiscovery/0.4 |
|
Domain checking tool |
31-mar-06 |
01-mag-05 |
| semanticdiscovery/2.0 |
|
Domain checking tool |
30-set-07 |
01-feb-07 |
| Sensis Web Crawler |
Sensis COM |
|
30-set-07 |
01-ott-05 |
| Setter Project |
|
|
30-set-07 |
01-ott-06 |
| shelob v1.0 |
|
Two things their researchers should take note of: The robots.txt standard, and the fact that shelob was an evil spider... at least according to Tolkien |
30-set-07 |
01-lug-07 |
| sherlock/1.0 |
|
MacOS 8.5 plug-in |
31-mar-05 |
01-dic-04 |
| Shim-Crawler |
|
|
30-set-07 |
01-nov-05 |
| ShowTags/1.0 libwww/5.4.0 |
|
|
31-gen-05 |
01-dic-04 |
| Sigram/Nutch-0.9-dev |
|
|
30-nov-06 |
01-nov-06 |
| Silk/1.0 |
|
|
30-nov-06 |
01-dic-05 |
| silk/2.4 |
|
|
31-lug-05 |
01-lug-05 |
| Sirketcebot/v.01 |
|
http://www.berilteknoloji.com/ |
30-set-07 |
01-giu-07 |
| SiteBus |
|
|
31-mag-04 |
01-mag-04 |
| SiteSnagger |
|
|
31-ott-05 |
01-ott-05 |
| SiteSpider |
|
Beta testing - dic 2004 motore non ancora attivo |
30-nov-04 |
01-nov-04 |
| SiteXpert |
|
Sitemap & search engine builder - http://www.xtreeme.com/sitexpert/index.php |
30-giu-04 |
01-giu-04 |
| SitiDi.net/SitiDiBot/1.0 |
|
|
30-set-05 |
01-apr-05 |
| Skywalker/0.1 |
|
|
31-mag-06 |
01-mag-06 |
| SlimBrowser |
|
|
30-giu-05 |
01-nov-04 |
| slurp |
|
|
30-apr-04 |
01-apr-04 |
| Slurp/0.8-dev |
|
|
30-nov-05 |
01-nov-05 |
| Slurp/2.0 |
|
Inktomi ( Hotbot, Snap etc.) |
30-giu-04 |
01-giu-04 |
| SMEALSearch-Bot |
|
|
31-mar-04 |
01-mar-04 |
| SMS Gateway |
|
|
30-set-07 |
01-nov-06 |
| snap.com beta crawler v0 |
|
|
30-apr-06 |
01-apr-05 |
| Snapbot/1.0 |
|
|
30-set-07 |
01-mag-06 |
| SnapPreviewBot |
|
|
31-ago-07 |
01-giu-07 |
| Snappy/1.1 |
|
|
30-set-07 |
01-lug-06 |
| snipsearch/Nutch-0.8 |
|
|
31-ago-06 |
01-ago-06 |
| Snoopy v0.92 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
30-giu-04 |
01-giu-04 |
| Snoopy v1.01 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
30-giu-06 |
01-apr-04 |
| Snoopy v1.2 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
31-ago-07 |
01-gen-07 |
| SOFT411 Directory http://www.soft411.com |
Soft411 COM |
|
31-ago-04 |
01-lug-04 |
| SOFT411 Directory http://www.soft411.com/ |
Soft411 COM |
|
30-nov-04 |
01-nov-04 |
| Sogou Push Spider/3.0 |
|
|
30-set-07 |
01-set-07 |
| sogou spider |
|
|
30-apr-07 |
01-feb-07 |
| sohu-search |
Sohu COM |
|
30-nov-05 |
01-lug-04 |
| Sopheus Project/0.01 |
Thenetplanet (COM) |
Self-service information mining tools that empower IT users to find, recycle, create and publish corporate knowledge |
31-mar-05 |
01-mar-05 |
| sp0003.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0004.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0006.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0009.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0026.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0035.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| sp0041.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
| SpaceBison/0.01 [fu] |
|
|
30-set-05 |
01-set-05 |
| Speedy Spider |
EntireWeb COM |
|
30-set-07 |
01-apr-04 |
| SpeedySpider |
Entireweb COM |
|
30-apr-05 |
01-feb-05 |
| Sphider |
|
|
30-apr-06 |
01-dic-05 |
| Spider Noago.it |
|
|
31-mar-04 |
01-feb-04 |
| Spider Test 2.4 |
|
? |
31-gen-04 |
01-gen-04 |
| SPIDER www.altracustica.org |
|
|
30-giu-04 |
01-giu-04 |
| SpiderMan |
|
|
31-mag-06 |
01-mag-05 |
| SpiderMan Mozilla/4.0 |
|
|
30-nov-06 |
01-set-06 |
| SpiderTest |
|
? |
31-gen-04 |
01-gen-04 |
| SplatSearch.com |
|
|
30-set-07 |
01-mar-06 |
| sproose/0.1 |
Sproose COM |
|
30-set-06 |
01-giu-06 |
| sproose/0.1-alpha |
Sproose COM |
|
30-giu-06 |
01-apr-06 |
| sproose/1.0beta |
Sproose COM |
|
30-set-07 |
01-ott-06 |
| SpurlBot/0.3 |
|
|
31-lug-05 |
01-lug-05 |
| SquidClamAV_Redirector 1.6 |
|
|
31-mag-05 |
01-mag-05 |
| SquidClamAV_Redirector 1.6.3 |
|
|
31-ago-07 |
01-mar-06 |
| SquidClamAV_Redirector 1.7.0 |
|
|
31-mag-06 |
01-nov-05 |
| Squid-Prefetch |
|
|
30-set-07 |
01-gen-06 |
| Sqworm/2.9.85-BETA |
|
? diff. IPs services, Inria.fr robot -Websense (Internet filtering) robot, AOL Search / Pacific Internet Exchange robot |
31-ago-04 |
01-feb-04 |
| SrevBot/2.0 |
|
|
31-mag-06 |
01-mag-06 |
| SSM Agent 1.0 |
|
? program that will download a list of webpages using the winsock control |
30-set-04 |
01-gen-04 |
| StackRambler/2.0 |
|
|
31-gen-05 |
01-dic-04 |
| Star Downloader |
|
Downloader |
31-mag-05 |
01-set-04 |
| StarDownloader/1.44 |
|
|
28-feb-05 |
01-feb-05 |
| StarOffice/5.2 |
|
|
28-feb-05 |
01-feb-05 |
| stat |
|
|
31-ott-04 |
01-ott-04 |
| stat statcrawler@gmail.com |
|
Experimental search engine spider from 66.92.186.xxx |
31-ott-04 |
01-ott-04 |
| Steeler/2.0 |
|
University of Tokyo, Kitsuregawa Laboratory |
30-giu-04 |
01-mag-04 |
| STEROID Download |
|
|
31-mar-06 |
01-dic-05 |
| SuperCleaner 2.57 |
|
|
31-ago-04 |
01-ago-04 |
| SuperGet/0.1 |
|
? (Ideare) |
31-mag-04 |
01-gen-04 |
| SURF |
|
Content filtering software - www.surfcontrol.com |
31-lug-04 |
01-mar-04 |
| SurveyBot/2.3 |
|
Monitors Internet StatisticsEach week SurveyBot will query websites for statistics and other useful information. This information goes into the creation of the Whois Source domain search engine (www.whois.sc advertisment robot) |
30-set-04 |
01-feb-04 |
| SygolBot |
|
|
30-set-07 |
01-lug-07 |
| SygolBot http://www.sygol.com |
|
Motore italiano, domini, scambio banner, ecc. |
30-set-07 |
01-mar-06 |
| SygolBot http://www.sygol.it |
|
Motore italiano, domini, scambio banner, ecc. |
30-set-07 |
01-set-07 |
| SygolBot http://www.sygol.net |
|
|
30-apr-06 |
01-ago-05 |
| SynooBot/0.7.1 |
|
|
30-apr-06 |
01-dic-05 |
| Synoobot/0.9 |
|
|
31-ago-07 |
01-apr-07 |
| Syntryx ANT Scout Chassis Pheromone |
|
|
31-lug-06 |
01-apr-06 |
| Szukacz/1.5 |
Szukacz (POLONIA) |
|
30-nov-06 |
01-mar-04 |
| TagTag emulator v1.12 |
|
|
30-giu-05 |
01-giu-05 |
| Taiga web spider |
|
|
30-nov-06 |
01-nov-06 |
| TALWinHttpClient |
|
|
31-ott-06 |
01-ott-06 |
| TAMU_CS_IRL_CRAWLER/1.0 |
|
Texas A&M University - Dept. of Computer Science crawler (server or link checking ?) |
30-nov-04 |
01-mag-04 |
| Tcl http client package 2.3 |
|
keep-alive connection, establishes a persistent connection by default |
31-ott-04 |
01-feb-04 |
| Tcl http client package 2.4.2 |
|
Tcl provides a portable scripting environment for Unix, Windows, and Macintosh that supports string processing and pattern matching, native file system access, shell-like control over other programs, TCP/IP networking, timers, and event-driven I/O. |
30-apr-06 |
01-gen-05 |
| td aoh opoxxwqdib sfpl |
|
|
31-ago-06 |
01-ago-06 |
| Technoratibot/0.6 |
|
Tcl provides a portable scripting environment for Unix, Windows, and Macintosh that supports string processing and pattern matching, native file system access, shell-like control over other programs, TCP/IP networking, timers, and event-driven I/O. |
30-apr-04 |
01-apr-04 |
| Teleport Pro/1.29 |
|
Offline Browsing Webspider |
30-nov-06 |
01-apr-04 |
| Teleport Pro/1.29.1590 |
|
Offline Browsing Webspider |
31-ago-07 |
01-apr-04 |
| Teleport Pro/1.29.1718 |
|
Offline Browsing Webspider |
31-mag-04 |
01-mag-04 |
| Teleport Ultra/1.29.2052 |
|
Offline Browsing Webspider |
30-giu-04 |
01-giu-04 |
| tellbaby/Nutch-0.9 |
|
|
31-ago-07 |
01-apr-07 |
| tellbaby/Nutch-1.0-dev |
|
|
30-set-07 |
01-apr-07 |
| Teoma |
|
|
30-set-05 |
01-set-05 |
| teoma_agent1 |
Teoma COM |
? Unknown robot visiting pages and tacking "%09182837231" or somesuch onto the ends of URL's |
30-set-07 |
01-mar-04 |
| TerrawizBot/1.0 |
|
|
31-ago-06 |
01-lug-06 |
| test/0.1 |
|
|
31-mar-05 |
01-mar-04 |
| test/Nutch-0.8.1 |
|
|
31-ott-06 |
01-ott-06 |
| Test_Robot_1.1/Virem |
|
|
30-apr-06 |
01-gen-06 |
| TestCrawler/Nutch-0.9 |
|
|
30-set-07 |
01-ago-07 |
| testnutch/Nutch-0.9 |
|
|
31-ago-07 |
01-ago-07 |
| testnutch/Nutch-1.0-dev |
|
|
30-set-07 |
01-set-07 |
| TestSQLLite |
|
|
31-ago-04 |
01-ago-04 |
| tfpfbetypjtnhdbcveWu t7oWuyy |
|
|
30-nov-06 |
01-nov-06 |
| Theophrastus/1.2 |
|
|
30-apr-06 |
01-dic-05 |
| Theophrastus/2.1 |
|
|
30-apr-06 |
01-gen-06 |
| TheSpireProject_squirrel |
|
|
30-giu-06 |
01-apr-06 |
| Thomas Krichel |
|
|
30-apr-07 |
01-gen-07 |
| Thumbnail.CZ robot 1.0 |
|
|
31-ott-05 |
01-set-05 |
| Thumbnail.CZ robot 1.1 |
|
|
31-mar-06 |
01-dic-05 |
| T-H-U-N-D-E-R-S-T-O-N-E |
Thunderstone COM |
|
30-set-06 |
01-feb-04 |
| TMCrawler |
|
|
30-set-07 |
01-set-06 |
| T-Online Browser |
|
|
30-set-07 |
01-apr-07 |
| Trend Micro tmdr 1.0-1000 |
|
|
30-nov-05 |
01-apr-05 |
| Trend Micro tmdr 1.0-1032 |
|
|
31-mar-05 |
01-feb-05 |
| Trend Micro tmdr 1.0-1110 |
|
|
31-mar-05 |
01-feb-05 |
| Trend Micro tmdr 1.0-1139 |
|
|
30-nov-05 |
01-mag-05 |
| Trend Micro tmdr 1.2-1003 |
|
|
30-set-06 |
01-feb-06 |
| trexmod |
|
? |
31-ago-04 |
01-mag-04 |
| TridentSpider/3.1 |
|
|
31-ott-06 |
01-ott-06 |
| troovziBot |
|
|
31-mag-06 |
01-mag-06 |
| TulipChain/6.03 |
|
Browser / link checker for Dmoz.org directory |
31-dic-04 |
01-dic-04 |
| TurnitinBot/1.5 |
|
To help educational institutions prevent plagiarism |
31-mar-04 |
01-gen-04 |
| TurnitinBot/1.5 http://www.turnitin.com |
|
To help educational institutions prevent plagiarism |
31-mar-04 |
01-gen-04 |
| TurnitinBot/2.0 |
|
To help educational institutions prevent plagiarism |
30-nov-04 |
01-mar-04 |
| TurnitinBot/2.0 http://www.turnitin.com |
|
To help educational institutions prevent plagiarism |
30-nov-04 |
01-mar-04 |
| TutorGig/1.5 |
|
To help educational institutions prevent plagiarism |
30-set-04 |
01-ago-04 |
| TutorGigBot/1.5 |
|
? Only indexing sites having tutorials, guides, learning material on their site |
30-nov-04 |
01-nov-04 |
| Tutorial Crawler 1.4 |
|
Only indexing sites having tutorials, guides, learning material on their site |
31-ago-04 |
01-mar-04 |
| TuttonetBot/1.1 |
TuttoNet (IT) |
www.tuttonet.com |
31-lug-04 |
01-mag-04 |
| Twiceler www.cuill.com/robots.html |
|
experimental web crawler - costello@cs.stanford.edu |
31-ago-07 |
01-mag-05 |
| Twiceler www.cuill.com |
|
Experimental robot |
31-ago-07 |
01-dic-06 |
| Twiceler-0.9 http://www.cuill.com |
|
experimental robot. Please contact costello@cuill.com if you have any problems. Twiceler should obey robots.txt. |
31-ago-07 |
01-apr-07 |
| Twisted PageGetter |
|
|
30-apr-07 |
01-feb-07 |
| TygoBot |
Tygo COM |
|
30-set-04 |
01-mar-04 |
| TygoProwler |
Tygo COM |
? |
31-gen-05 |
01-nov-04 |
| U |
|
? |
30-giu-07 |
01-gen-04 |
| UbiCrawler/v0.4beta |
|
CNR Italia |
31-ago-04 |
01-feb-04 |
| UbiCrawler/v0.5beta |
|
|
31-mag-06 |
01-mag-06 |
| UdmSearch/3.1.19 |
|
Offline browser/search client |
31-ago-04 |
01-ago-04 |
| Ultraseek |
|
|
31-mar-05 |
01-dic-04 |
| Under the Rainbow 2.2 |
|
|
31-lug-05 |
01-feb-05 |
| UniFind Site Spider |
|
|
31-ago-04 |
01-mag-04 |
| UniversalFeedParser/3.0-beta-19 |
|
http://diveintomark.org/projects/feed_parser/ |
30-apr-04 |
01-apr-04 |
| University of Missouri Web |
|
|
31-mar-04 |
01-mar-04 |
| unknown/1.0 |
|
|
30-set-07 |
01-mag-04 |
| UofTDB_experiment |
|
|
31-lug-05 |
01-lug-05 |
| UofTDB_experiment leehyun@cs.toronto.edu |
|
|
31-lug-05 |
01-lug-05 |
| updated.com/1beta |
|
? Clone |
31-gen-05 |
01-dic-04 |
| updated/0.1-alpha |
|
|
30-giu-06 |
01-giu-06 |
| updated/0.1beta |
|
|
30-set-05 |
01-gen-05 |
| updated/0.2-dev |
|
|
31-ott-05 |
01-ott-05 |
| UptimeBot |
|
Servizio online di monitoraggio siti, check links - www.uptimebot.com |
31-lug-04 |
01-mar-04 |
| Urhebersuche |
|
|
30-set-07 |
01-lug-07 |
| URI::Fetch/0.08 |
|
|
30-set-07 |
01-feb-07 |
| url checker: biome.ac.uk |
|
|
31-ago-04 |
01-mar-04 |
| User-Agent |
|
|
30-giu-07 |
01-giu-07 |
| User-Agent: BoardReader-Image-Fetcher /1.0 info@boardreader.com |
|
|
30-set-07 |
01-ago-07 |
| User-Agent: Mozilla/4.0 |
|
|
31-mar-06 |
01-dic-05 |
| Uywrxxuxnjdh2filita f |
|
|
31-mag-07 |
01-mag-07 |
| Vagabondo/2.0 MT |
|
One or more search engines, maintained by WiseGuys |
30-nov-04 |
01-mar-04 |
| Vagabondo/2.2 |
|
One or more search engines, maintained by WiseGuys |
31-mag-05 |
01-mar-04 |
| Vagabondo/2.3 |
|
One or more search engines, maintained by WiseGuys |
31-ago-07 |
01-lug-05 |
| Vagabondo/3.0 |
|
One or more search engines, maintained by WiseGuys |
31-ago-07 |
01-mar-05 |
| VB L@n Backup Live Update |
|
|
30-apr-06 |
01-dic-05 |
| VerbaDCSBot/1.0 http://wwww.esand.net |
|
|
31-gen-05 |
01-dic-04 |
| Verify |
|
|
30-giu-04 |
01-giu-04 |
| vietnamnet/Nutch-0.9 |
|
|
30-set-07 |
01-set-07 |
| VirgilioBot |
Virgilio IT |
|
30-set-07 |
01-feb-06 |
| virus_detector |
|
|
31-mar-06 |
01-mar-06 |
| virus_detector |
|
virus_harvester@securecomputing.com |
31-mar-06 |
01-mar-06 |
| Visbot/1.0 |
|
|
30-set-06 |
01-set-06 |
| Visbot/1.1 |
|
|
30-nov-06 |
01-nov-06 |
| VisBot/2.0 |
|
|
30-set-07 |
01-apr-07 |
| VisWeb |
|
|
29-feb-04 |
01-feb-04 |
| VORTEX/1.0 |
|
|
30-apr-06 |
01-dic-05 |
| VORTEX/1.2 |
|
|
30-apr-06 |
01-dic-05 |
| Vortex/2.2 |
|
|
30-apr-06 |
01-dic-05 |
| Voyager |
|
|
31-ott-05 |
01-lug-05 |
| voyager/1.0 |
|
hosted by Cosmix Corp. -"a start-up based on novel and fundamental algorithm and system research"- |
30-set-07 |
01-nov-05 |
| VSE/1.0 |
|
? hotmail.com |
30-nov-05 |
01-feb-04 |
