Skip to search formSkip to main contentSkip to account menu

Web crawler

Known as: Webcrawler, Crawl site, RBSE 
A Web crawler is an Internet bot which systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). Web… 
Wikipedia (opens in a new tab)

Related topics

Related topics

50 relations
Ajax (programming)Apache HadoopApache NutchApache Storm

Papers overview

Semantic Scholar uses AI to extract papers important to this topic.
Highly Cited
2012
Highly Cited
2012
A Study of Recommending Locations on Location-Based Social Network by Collaborative Filtering
The development of location-based social networking (LBSN) services is growing rapidly these days. Users of LBSN services are… 
2007
2007
Semantic deep web: automatic attribute extraction from the deep web data sources
"Deep Web" refers to the rich information and data hidden in backend databases, etc., that search engines or Web crawlers cannot… 
Highly Cited
2006
Highly Cited
2006
Malware prevalence in the KaZaA file-sharing network
In recent years, more than 200 viruses have been reported to use a peer-to-peer (P2P) file-sharing network as a propagation… 
Highly Cited
2005
Highly Cited
2005
DR-NEGOTIATE - a system for automated agent negotiation with defeasible logic-based strategies
Highly Cited
2005
Highly Cited
2005
Constructing Interface Schemas for Search Interfaces of Web Databases
Many databases have become Web-accessible through form-based search interfaces (i.e., search forms) that allow users to specify… 
Highly Cited
2005
Highly Cited
2005
Understanding How Spammers Steal Your E-Mail Address: An Analysis of the First Six Months of Data from Project Honey Pot
This paper summarizes and analyses data compiled on the activities of email harvesters gathered through a 5,000+ member honey pot… 
Highly Cited
2005
Highly Cited
2005
What's there and what's not?: focused crawling for missing documents in digital libraries
Some large scale topical digital libraries, such as CiteSeer, harvest online academic documents by crawling open-access archives… 
Highly Cited
2004
Highly Cited
2004
A functional relationship between species richness of spiders and lichens in spruce
Modern forestry has created stands with even age distribution of trees and fragmentation of the habitat. In boreal forests, the… 
Highly Cited
2003
Highly Cited
2003
Development of "Souryu I & II" -Connected Crawler Vehicle for Inspection of Narrow and Winding Space
Highly Cited
1979
Highly Cited
1979
Games spiders play
Summary1.The predictions of game theory concerning the use of resource assessment strategies by animals in conflict situations… 
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)