This paper discusses the design and performance of a hierarchical proxy-cache designed to make Internet information systems scale better. The design was motivated by our earlier trace-driven simulation study of Internet traffic. We believe that the conventional wisdom, that the benefits of hierarchical file caching do not merit the costs, warrants… (More)
It is increasingly difficult to make effective use of Internet information, given the rapid growth in data volume, user base, and data diversity. In this paper we introduce Harvest, a system that provides a scalable, customizable architecture for gathering, indexing, caching, replicat-ing, and accessing Internet information.
This paper presents evidence that several, judiciously placed file caches could reduce the volume of FTP traffic by 42%, and hence the volume of all NSFNET backbone traffic by 21%. In addition, if FTP client and server software automatically compressed data, this savings could increase to 27%. We believe that a hierarchical architecture of whole file… (More)
ngoing increases in wide-area network connectivity promise vastly increased opportunities for collaboration and resource sharing. A fundamental problem confronting users of such networks i,; how to discover the existence of resources of interest, such as files, retail products, network services, or people. In tZhis article we focus on the problem of… (More)
In the past several years, the number and variety of resources available on the Internet have increased dramatically. With this increase, many new systems have been developed that allow users to search for and access these resources. As these systems begin to interconnect with one another through "information gate-ways", the conceptual relationships between… (More)
Rapidly increasing global Internet connectivity offers tremendous opportunities for collaboration and information sharing. An important problem in this environment is how to discover resources of interest, such as documents, network services, and people. In this paper we discuss a number of aspects of the resource discovery problem, and summarize results… (More)
Wide area networks offer access to an increasing number and variety of resources, such as documents, software, and network services. Yet, it is difficult to locate resources of interest, because of the scale and decentralized nature of the environment. We are interested in supporting a global confederation of loosely cooperating systems and users that share… (More)
In this paper we consider the problem of choosing among a collection of replicated servers, focusing on the question of how to make choices that segregate client/server traffic according to network topology. We explore the cost and effectiveness of a variety of approaches, ranging from those requiring routing layer support (e.g., anycast) to those that… (More)