Skip to search formSkip to main content
You are currently offline. Some features of the site may not work correctly.

Distributed web crawling

Known as: Distributed crawling, Distributed search, Distributed web crawler 
Distributed web crawling is a distributed computing technique whereby Internet search engines employ many computers to index the Internet via web… Expand
Wikipedia

Papers overview

Semantic Scholar uses AI to extract papers important to this topic.
Highly Cited
2004
Highly Cited
2004
We report our experience in implementing UbiCrawler, a scalable distributed Web crawler, using the Java programming language. The… Expand
Is this relevant?
2004
2004
In this paper, we present the design and implementation of a distributed web crawler. We begin by motivating the need for such a… Expand
  • figure 1
  • figure 2
  • figure 4
  • figure 3
  • figure 5
Is this relevant?
Highly Cited
2003
Highly Cited
2003
The recent file storage applications built on top of peer-to-peer distributed hash tables lack search capabilities. We believe… Expand
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • figure 5
Is this relevant?
Highly Cited
2003
Highly Cited
2003
A super-peer is a node in a peer-to-peer network that operates both as a server to a set of clients, and as an equal in a network… Expand
  • figure 1
  • table 1
  • table 2
  • table 3
  • figure 2
Is this relevant?
Highly Cited
2002
Highly Cited
2002
Broad Web search engines as well as many more specialized search tools rely on Web crawlers to acquire large collections of pages… Expand
  • figure 1
  • figure 2
  • figure 3
  • figure 4
  • table 4.1
Is this relevant?
Highly Cited
2002
Highly Cited
2002
  • Sam Joseph
  • NETWORKING Workshops
  • 2002
  • Corpus ID: 15404038
NeuroGrid is an adaptive decentralized search system. NeuroGrid nodes support distributed search through semantic routing… Expand
  • figure 1
  • figure 2
  • figure 2
Is this relevant?
Highly Cited
1991
Highly Cited
1991
  • V. Lesser
  • IEEE Trans. Syst. Man Cybern.
  • 1991
  • Corpus ID: 9314935
The functionally accurate, cooperative (FA/C) paradigm provides a model for task decomposition and agent interaction in a… Expand
  • figure 1
  • figure 2
  • figure 3
  • figure 4
Is this relevant?
Highly Cited
1991
Highly Cited
1991
It is theorized that the process of coordination is a distributed search through a hierarchical space of agent behaviors. By… Expand
Is this relevant?
Highly Cited
1990
Highly Cited
1990
Abstract GENITOR is a genetic algorithm which employs one-at-a-time reproduction and allocates reproductive opportunities… Expand
  • figure I
  • figure 2
  • figure 3
Is this relevant?
Highly Cited
1981
Highly Cited
1981
A new approach for structuring distributed processing systems, called functionally accurate, cooperative (FA/C), is proposed. The… Expand
  • table 1
Is this relevant?