sait/Nutch-0.9 |
|
|
30-set-07 |
01-set-07 |
sallatiBOT_sallati.comV1.0 |
|
Web development & SEO - http://www.sallati.com |
30-set-04 |
01-set-04 |
savvybot/0.2 |
|
|
31-mar-05 |
01-mar-05 |
SBIder/0.7 |
Sitesell COM |
|
30-set-05 |
01-set-05 |
SBIder/0.8-dev |
Sitesell COM |
|
30-apr-06 |
01-set-05 |
SboongBot |
Sboong.com/it |
Estrae indirizzi e-mail da pagine web e da usenet per alimentare il DB del motore di ricerca e-mail |
30-giu-04 |
01-mar-04 |
ScanWebBot/1.0 |
|
|
30-set-06 |
01-giu-06 |
Schmozilla/v9.14 Platinum |
|
? Fetching a URL from a Perl Script |
30-giu-05 |
01-feb-04 |
Scooter/2.0 |
Altavista COM |
non attivo dal 07/04 |
31-lug-04 |
01-lug-04 |
Scooter/3.3 |
Altavista COM |
|
30-set-07 |
01-gen-04 |
Scooter/3.3.vscooter |
Altavista COM |
non attivo dal 03/04 |
31-mar-04 |
01-gen-04 |
Scooter/3.3_SF |
Altavista COM |
non attivo dal 04/04 |
30-apr-04 |
01-gen-04 |
Scooter/3.3Y!CrawlX |
Altavista COM |
non attivo dal 08/04 |
31-ago-04 |
01-ago-04 |
Scorpion V1.0 Crawler |
|
|
31-ago-04 |
01-ago-04 |
Scrubby/2.1 |
ScrubTheWeb COM |
|
29-feb-04 |
01-feb-04 |
Scrubby/2.2 |
ScrubTheWeb COM |
|
30-set-07 |
01-feb-04 |
Scrubby/3.0 |
ScrubTheWeb COM |
|
30-set-07 |
01-ago-07 |
ScSpider/0.2 |
|
|
30-set-07 |
01-lug-04 |
Scumbot/5.5 |
|
|
30-apr-06 |
01-apr-06 |
Search Engine World Robots.txt Validator at http://www.searchengineworld.com |
|
searchengineworld.com |
31-ott-04 |
01-ott-04 |
Search Fst |
|
|
30-giu-04 |
01-giu-04 |
search.updated.com/0.06 |
Updated COM |
|
30-nov-04 |
01-nov-04 |
SearchBlox |
|
|
30-set-07 |
01-ago-07 |
Search-Channel |
SearchChannell (FRANCIA) |
Adult Search Channel |
31-gen-05 |
01-apr-04 |
searchengineBot 1.1 |
|
? searchenginebot.com |
30-nov-04 |
01-nov-04 |
SearchGuild DMOZ Experiment |
|
|
31-ago-05 |
01-ago-05 |
SearchSight/2.0 |
|
|
30-giu-06 |
01-ott-05 |
SearchSight_Crawler/2.0 |
|
|
30-set-05 |
01-set-05 |
SearchWarp_Crawler/2.0 |
|
|
31-ago-05 |
01-ago-05 |
SEB Spider |
|
|
31-ott-05 |
01-set-05 |
SecretBrowser/007 |
|
|
31-ago-05 |
01-mag-04 |
Seekbot/1.0 |
Seekbot NET |
Germania |
30-set-07 |
01-ago-04 |
Seekbot/2.x |
|
|
30-set-05 |
01-set-05 |
Selflinkchecker 1.0 |
|
|
31-mag-06 |
01-mag-06 |
semanticdiscovery/0.2 |
|
Domain checking tool |
31-lug-04 |
01-gen-04 |
semanticdiscovery/0.3 |
|
Domain checking tool |
30-nov-04 |
01-lug-04 |
semanticdiscovery/0.4 |
|
Domain checking tool |
31-mar-06 |
01-mag-05 |
semanticdiscovery/2.0 |
|
Domain checking tool |
30-set-07 |
01-feb-07 |
Sensis Web Crawler |
Sensis COM |
|
30-set-07 |
01-ott-05 |
Setter Project |
|
|
30-set-07 |
01-ott-06 |
shelob v1.0 |
|
Two things their researchers should take note of: The robots.txt standard, and the fact that shelob was an evil spider... at least according to Tolkien |
30-set-07 |
01-lug-07 |
sherlock/1.0 |
|
MacOS 8.5 plug-in |
31-mar-05 |
01-dic-04 |
Shim-Crawler |
|
|
30-set-07 |
01-nov-05 |
ShowTags/1.0 libwww/5.4.0 |
|
|
31-gen-05 |
01-dic-04 |
Sigram/Nutch-0.9-dev |
|
|
30-nov-06 |
01-nov-06 |
Silk/1.0 |
|
|
30-nov-06 |
01-dic-05 |
silk/2.4 |
|
|
31-lug-05 |
01-lug-05 |
Sirketcebot/v.01 |
|
http://www.berilteknoloji.com/ |
30-set-07 |
01-giu-07 |
SiteBus |
|
|
31-mag-04 |
01-mag-04 |
SiteSnagger |
|
|
31-ott-05 |
01-ott-05 |
SiteSpider |
|
Beta testing - dic 2004 motore non ancora attivo |
30-nov-04 |
01-nov-04 |
SiteXpert |
|
Sitemap & search engine builder - http://www.xtreeme.com/sitexpert/index.php |
30-giu-04 |
01-giu-04 |
SitiDi.net/SitiDiBot/1.0 |
|
|
30-set-05 |
01-apr-05 |
Skywalker/0.1 |
|
|
31-mag-06 |
01-mag-06 |
SlimBrowser |
|
|
30-giu-05 |
01-nov-04 |
slurp |
|
|
30-apr-04 |
01-apr-04 |
Slurp/0.8-dev |
|
|
30-nov-05 |
01-nov-05 |
Slurp/2.0 |
|
Inktomi ( Hotbot, Snap etc.) |
30-giu-04 |
01-giu-04 |
SMEALSearch-Bot |
|
|
31-mar-04 |
01-mar-04 |
SMS Gateway |
|
|
30-set-07 |
01-nov-06 |
snap.com beta crawler v0 |
|
|
30-apr-06 |
01-apr-05 |
Snapbot/1.0 |
|
|
30-set-07 |
01-mag-06 |
SnapPreviewBot |
|
|
31-ago-07 |
01-giu-07 |
Snappy/1.1 |
|
|
30-set-07 |
01-lug-06 |
snipsearch/Nutch-0.8 |
|
|
31-ago-06 |
01-ago-06 |
Snoopy v0.92 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
30-giu-04 |
01-giu-04 |
Snoopy v1.01 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
30-giu-06 |
01-apr-04 |
Snoopy v1.2 |
|
PHP class that simulates a web browser. It automates the task of retrieving web page content and posting forms |
31-ago-07 |
01-gen-07 |
SOFT411 Directory http://www.soft411.com |
Soft411 COM |
|
31-ago-04 |
01-lug-04 |
SOFT411 Directory http://www.soft411.com/ |
Soft411 COM |
|
30-nov-04 |
01-nov-04 |
Sogou Push Spider/3.0 |
|
|
30-set-07 |
01-set-07 |
sogou spider |
|
|
30-apr-07 |
01-feb-07 |
sohu-search |
Sohu COM |
|
30-nov-05 |
01-lug-04 |
Sopheus Project/0.01 |
Thenetplanet (COM) |
Self-service information mining tools that empower IT users to find, recycle, create and publish corporate knowledge |
31-mar-05 |
01-mar-05 |
sp0003.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0004.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0006.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0009.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0026.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0035.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
sp0041.jiffe.com |
|
|
30-giu-04 |
01-giu-04 |
SpaceBison/0.01 [fu] |
|
|
30-set-05 |
01-set-05 |
Speedy Spider |
EntireWeb COM |
|
30-set-07 |
01-apr-04 |
SpeedySpider |
Entireweb COM |
|
30-apr-05 |
01-feb-05 |
Sphider |
|
|
30-apr-06 |
01-dic-05 |
Spider Noago.it |
|
|
31-mar-04 |
01-feb-04 |
Spider Test 2.4 |
|
? |
31-gen-04 |
01-gen-04 |
SPIDER www.altracustica.org |
|
|
30-giu-04 |
01-giu-04 |
SpiderMan |
|
|
31-mag-06 |
01-mag-05 |
SpiderMan Mozilla/4.0 |
|
|
30-nov-06 |
01-set-06 |
SpiderTest |
|
? |
31-gen-04 |
01-gen-04 |
SplatSearch.com |
|
|
30-set-07 |
01-mar-06 |
sproose/0.1 |
Sproose COM |
|
30-set-06 |
01-giu-06 |
sproose/0.1-alpha |
Sproose COM |
|
30-giu-06 |
01-apr-06 |
sproose/1.0beta |
Sproose COM |
|
30-set-07 |
01-ott-06 |
SpurlBot/0.3 |
|
|
31-lug-05 |
01-lug-05 |
SquidClamAV_Redirector 1.6 |
|
|
31-mag-05 |
01-mag-05 |
SquidClamAV_Redirector 1.6.3 |
|
|
31-ago-07 |
01-mar-06 |
SquidClamAV_Redirector 1.7.0 |
|
|
31-mag-06 |
01-nov-05 |
Squid-Prefetch |
|
|
30-set-07 |
01-gen-06 |
Sqworm/2.9.85-BETA |
|
? diff. IPs services, Inria.fr robot -Websense (Internet filtering) robot, AOL Search / Pacific Internet Exchange robot |
31-ago-04 |
01-feb-04 |
SrevBot/2.0 |
|
|
31-mag-06 |
01-mag-06 |
SSM Agent 1.0 |
|
? program that will download a list of webpages using the winsock control |
30-set-04 |
01-gen-04 |
StackRambler/2.0 |
|
|
31-gen-05 |
01-dic-04 |
Star Downloader |
|
Downloader |
31-mag-05 |
01-set-04 |
StarDownloader/1.44 |
|
|
28-feb-05 |
01-feb-05 |
StarOffice/5.2 |
|
|
28-feb-05 |
01-feb-05 |
stat |
|
|
31-ott-04 |
01-ott-04 |
stat statcrawler@gmail.com |
|
Experimental search engine spider from 66.92.186.xxx |
31-ott-04 |
01-ott-04 |
Steeler/2.0 |
|
University of Tokyo, Kitsuregawa Laboratory |
30-giu-04 |
01-mag-04 |
STEROID Download |
|
|
31-mar-06 |
01-dic-05 |
SuperCleaner 2.57 |
|
|
31-ago-04 |
01-ago-04 |
SuperGet/0.1 |
|
? (Ideare) |
31-mag-04 |
01-gen-04 |
SURF |
|
Content filtering software - www.surfcontrol.com |
31-lug-04 |
01-mar-04 |
SurveyBot/2.3 |
|
Monitors Internet StatisticsEach week SurveyBot will query websites for statistics and other useful information. This information goes into the creation of the Whois Source domain search engine (www.whois.sc advertisment robot) |
30-set-04 |
01-feb-04 |
SygolBot |
|
|
30-set-07 |
01-lug-07 |
SygolBot http://www.sygol.com |
|
Motore italiano, domini, scambio banner, ecc. |
30-set-07 |
01-mar-06 |
SygolBot http://www.sygol.it |
|
Motore italiano, domini, scambio banner, ecc. |
30-set-07 |
01-set-07 |
SygolBot http://www.sygol.net |
|
|
30-apr-06 |
01-ago-05 |
SynooBot/0.7.1 |
|
|
30-apr-06 |
01-dic-05 |
Synoobot/0.9 |
|
|
31-ago-07 |
01-apr-07 |
Syntryx ANT Scout Chassis Pheromone |
|
|
31-lug-06 |
01-apr-06 |
Szukacz/1.5 |
Szukacz (POLONIA) |
|
30-nov-06 |
01-mar-04 |
TagTag emulator v1.12 |
|
|
30-giu-05 |
01-giu-05 |
Taiga web spider |
|
|
30-nov-06 |
01-nov-06 |
TALWinHttpClient |
|
|
31-ott-06 |
01-ott-06 |
TAMU_CS_IRL_CRAWLER/1.0 |
|
Texas A&M University - Dept. of Computer Science crawler (server or link checking ?) |
30-nov-04 |
01-mag-04 |
Tcl http client package 2.3 |
|
keep-alive connection, establishes a persistent connection by default |
31-ott-04 |
01-feb-04 |
Tcl http client package 2.4.2 |
|
Tcl provides a portable scripting environment for Unix, Windows, and Macintosh that supports string processing and pattern matching, native file system access, shell-like control over other programs, TCP/IP networking, timers, and event-driven I/O. |
30-apr-06 |
01-gen-05 |
td aoh opoxxwqdib sfpl |
|
|
31-ago-06 |
01-ago-06 |
Technoratibot/0.6 |
|
Tcl provides a portable scripting environment for Unix, Windows, and Macintosh that supports string processing and pattern matching, native file system access, shell-like control over other programs, TCP/IP networking, timers, and event-driven I/O. |
30-apr-04 |
01-apr-04 |
Teleport Pro/1.29 |
|
Offline Browsing Webspider |
30-nov-06 |
01-apr-04 |
Teleport Pro/1.29.1590 |
|
Offline Browsing Webspider |
31-ago-07 |
01-apr-04 |
Teleport Pro/1.29.1718 |
|
Offline Browsing Webspider |
31-mag-04 |
01-mag-04 |
Teleport Ultra/1.29.2052 |
|
Offline Browsing Webspider |
30-giu-04 |
01-giu-04 |
tellbaby/Nutch-0.9 |
|
|
31-ago-07 |
01-apr-07 |
tellbaby/Nutch-1.0-dev |
|
|
30-set-07 |
01-apr-07 |
Teoma |
|
|
30-set-05 |
01-set-05 |
teoma_agent1 |
Teoma COM |
? Unknown robot visiting pages and tacking "%09182837231" or somesuch onto the ends of URL's |
30-set-07 |
01-mar-04 |
TerrawizBot/1.0 |
|
|
31-ago-06 |
01-lug-06 |
test/0.1 |
|
|
31-mar-05 |
01-mar-04 |
test/Nutch-0.8.1 |
|
|
31-ott-06 |
01-ott-06 |
Test_Robot_1.1/Virem |
|
|
30-apr-06 |
01-gen-06 |
TestCrawler/Nutch-0.9 |
|
|
30-set-07 |
01-ago-07 |
testnutch/Nutch-0.9 |
|
|
31-ago-07 |
01-ago-07 |
testnutch/Nutch-1.0-dev |
|
|
30-set-07 |
01-set-07 |
TestSQLLite |
|
|
31-ago-04 |
01-ago-04 |
tfpfbetypjtnhdbcveWu t7oWuyy |
|
|
30-nov-06 |
01-nov-06 |
Theophrastus/1.2 |
|
|
30-apr-06 |
01-dic-05 |
Theophrastus/2.1 |
|
|
30-apr-06 |
01-gen-06 |
TheSpireProject_squirrel |
|
|
30-giu-06 |
01-apr-06 |
Thomas Krichel |
|
|
30-apr-07 |
01-gen-07 |
Thumbnail.CZ robot 1.0 |
|
|
31-ott-05 |
01-set-05 |
Thumbnail.CZ robot 1.1 |
|
|
31-mar-06 |
01-dic-05 |
T-H-U-N-D-E-R-S-T-O-N-E |
Thunderstone COM |
|
30-set-06 |
01-feb-04 |
TMCrawler |
|
|
30-set-07 |
01-set-06 |
T-Online Browser |
|
|
30-set-07 |
01-apr-07 |
Trend Micro tmdr 1.0-1000 |
|
|
30-nov-05 |
01-apr-05 |
Trend Micro tmdr 1.0-1032 |
|
|
31-mar-05 |
01-feb-05 |
Trend Micro tmdr 1.0-1110 |
|
|
31-mar-05 |
01-feb-05 |
Trend Micro tmdr 1.0-1139 |
|
|
30-nov-05 |
01-mag-05 |
Trend Micro tmdr 1.2-1003 |
|
|
30-set-06 |
01-feb-06 |
trexmod |
|
? |
31-ago-04 |
01-mag-04 |
TridentSpider/3.1 |
|
|
31-ott-06 |
01-ott-06 |
troovziBot |
|
|
31-mag-06 |
01-mag-06 |
TulipChain/6.03 |
|
Browser / link checker for Dmoz.org directory |
31-dic-04 |
01-dic-04 |
TurnitinBot/1.5 |
|
To help educational institutions prevent plagiarism |
31-mar-04 |
01-gen-04 |
TurnitinBot/1.5 http://www.turnitin.com |
|
To help educational institutions prevent plagiarism |
31-mar-04 |
01-gen-04 |
TurnitinBot/2.0 |
|
To help educational institutions prevent plagiarism |
30-nov-04 |
01-mar-04 |
TurnitinBot/2.0 http://www.turnitin.com |
|
To help educational institutions prevent plagiarism |
30-nov-04 |
01-mar-04 |
TutorGig/1.5 |
|
To help educational institutions prevent plagiarism |
30-set-04 |
01-ago-04 |
TutorGigBot/1.5 |
|
? Only indexing sites having tutorials, guides, learning material on their site |
30-nov-04 |
01-nov-04 |
Tutorial Crawler 1.4 |
|
Only indexing sites having tutorials, guides, learning material on their site |
31-ago-04 |
01-mar-04 |
TuttonetBot/1.1 |
TuttoNet (IT) |
www.tuttonet.com |
31-lug-04 |
01-mag-04 |
Twiceler www.cuill.com/robots.html |
|
experimental web crawler - costello@cs.stanford.edu |
31-ago-07 |
01-mag-05 |
Twiceler www.cuill.com |
|
Experimental robot |
31-ago-07 |
01-dic-06 |
Twiceler-0.9 http://www.cuill.com |
|
experimental robot. Please contact costello@cuill.com if you have any problems. Twiceler should obey robots.txt. |
31-ago-07 |
01-apr-07 |
Twisted PageGetter |
|
|
30-apr-07 |
01-feb-07 |
TygoBot |
Tygo COM |
|
30-set-04 |
01-mar-04 |
TygoProwler |
Tygo COM |
? |
31-gen-05 |
01-nov-04 |
U |
|
? |
30-giu-07 |
01-gen-04 |
UbiCrawler/v0.4beta |
|
CNR Italia |
31-ago-04 |
01-feb-04 |
UbiCrawler/v0.5beta |
|
|
31-mag-06 |
01-mag-06 |
UdmSearch/3.1.19 |
|
Offline browser/search client |
31-ago-04 |
01-ago-04 |
Ultraseek |
|
|
31-mar-05 |
01-dic-04 |
Under the Rainbow 2.2 |
|
|
31-lug-05 |
01-feb-05 |
UniFind Site Spider |
|
|
31-ago-04 |
01-mag-04 |
UniversalFeedParser/3.0-beta-19 |
|
http://diveintomark.org/projects/feed_parser/ |
30-apr-04 |
01-apr-04 |
University of Missouri Web |
|
|
31-mar-04 |
01-mar-04 |
unknown/1.0 |
|
|
30-set-07 |
01-mag-04 |
UofTDB_experiment |
|
|
31-lug-05 |
01-lug-05 |
UofTDB_experiment leehyun@cs.toronto.edu |
|
|
31-lug-05 |
01-lug-05 |
updated.com/1beta |
|
? Clone |
31-gen-05 |
01-dic-04 |
updated/0.1-alpha |
|
|
30-giu-06 |
01-giu-06 |
updated/0.1beta |
|
|
30-set-05 |
01-gen-05 |
updated/0.2-dev |
|
|
31-ott-05 |
01-ott-05 |
UptimeBot |
|
Servizio online di monitoraggio siti, check links - www.uptimebot.com |
31-lug-04 |
01-mar-04 |
Urhebersuche |
|
|
30-set-07 |
01-lug-07 |
URI::Fetch/0.08 |
|
|
30-set-07 |
01-feb-07 |
url checker: biome.ac.uk |
|
|
31-ago-04 |
01-mar-04 |
User-Agent |
|
|
30-giu-07 |
01-giu-07 |
User-Agent: BoardReader-Image-Fetcher /1.0 info@boardreader.com |
|
|
30-set-07 |
01-ago-07 |
User-Agent: Mozilla/4.0 |
|
|
31-mar-06 |
01-dic-05 |
Uywrxxuxnjdh2filita f |
|
|
31-mag-07 |
01-mag-07 |
Vagabondo/2.0 MT |
|
One or more search engines, maintained by WiseGuys |
30-nov-04 |
01-mar-04 |
Vagabondo/2.2 |
|
One or more search engines, maintained by WiseGuys |
31-mag-05 |
01-mar-04 |
Vagabondo/2.3 |
|
One or more search engines, maintained by WiseGuys |
31-ago-07 |
01-lug-05 |
Vagabondo/3.0 |
|
One or more search engines, maintained by WiseGuys |
31-ago-07 |
01-mar-05 |
VB L@n Backup Live Update |
|
|
30-apr-06 |
01-dic-05 |
VerbaDCSBot/1.0 http://wwww.esand.net |
|
|
31-gen-05 |
01-dic-04 |
Verify |
|
|
30-giu-04 |
01-giu-04 |
vietnamnet/Nutch-0.9 |
|
|
30-set-07 |
01-set-07 |
VirgilioBot |
Virgilio IT |
|
30-set-07 |
01-feb-06 |
virus_detector |
|
|
31-mar-06 |
01-mar-06 |
virus_detector |
|
virus_harvester@securecomputing.com |
31-mar-06 |
01-mar-06 |
Visbot/1.0 |
|
|
30-set-06 |
01-set-06 |
Visbot/1.1 |
|
|
30-nov-06 |
01-nov-06 |
VisBot/2.0 |
|
|
30-set-07 |
01-apr-07 |
VisWeb |
|
|
29-feb-04 |
01-feb-04 |
VORTEX/1.0 |
|
|
30-apr-06 |
01-dic-05 |
VORTEX/1.2 |
|
|
30-apr-06 |
01-dic-05 |
Vortex/2.2 |
|
|
30-apr-06 |
01-dic-05 |
Voyager |
|
|
31-ott-05 |
01-lug-05 |
voyager/1.0 |
|
hosted by Cosmix Corp. -"a start-up based on novel and fundamental algorithm and system research"- |
30-set-07 |
01-nov-05 |
VSE/1.0 |
|
? hotmail.com |
30-nov-05 |
01-feb-04 |