IDES: An Internet Distance Estimation Service for Large Networks

@article{Mao2006IDESAI,
  title={IDES: An Internet Distance Estimation Service for Large Networks},
  author={Yun Mao and Lawrence K. Saul and Jonathan M. Smith},
  journal={IEEE Journal on Selected Areas in Communications},
  year={2006},
  volume={24},
  pages={2273-2284}
}
The responsiveness of networked applications is limited by communications delays, making network distance an important parameter in optimizing the choice of communications peers. Since accurate global snapshots are difficult and expensive to gather and maintain, it is desirable to use sampling techniques in the Internet to predict unknown network distances from a set of partially observed measurements. This paper makes three contributions. First, we present a model for representing and… 

Figures and Tables from this paper

Network distance prediction for enabling service-oriented applications over large-scale networks
TLDR
The basic ideas behind network distance prediction are illustrated, the current research work based on different criteria is categorized, and how different protocols work is illustrated, and their merits and drawbacks are discussed.
DMFSGD: A Decentralized Matrix Factorization Algorithm for Network Distance Prediction
TLDR
This paper formulates the prediction problem as matrix completion where the unknown entries in a pairwise distance matrix constructed from a network are to be predicted, and assumes that the distance matrix has low-rank characteristics, so the problem is solvable by low- rank approximation based on matrix factorization.
Network Latency Estimation for Personal Devices: A Matrix Completion Approach
TLDR
This paper proposes a new class of low-rank matrix completion algorithms, which predicts the missing entries in an extracted “network feature matrix” by iteratively minimizing a weighted Schatten- $p$ norm to approximate the rank.
Using the Last-Mile Model as a Distributed Scheme for Available Bandwidth Prediction
TLDR
This paper designs decentralised heuristics to compute the capacities of each node so as to minimise the prediction error in the last-mile model, and shows that these algorithms can achieve a competitive accuracy even with asymmetric and erroneous end-to-end measurement datasets.
Decentralized prediction of end-to-end network performance classes
TLDR
This paper proposes a decentralized approach based on Stochastic Gradient Descent to solve the class-based matrix completion problem of unknown entries of an incomplete matrix of pairwise measurements, and demonstrates its accuracy, its robustness against erroneous measurements and its usability on peer selection.
Poster : AIDE : An Accurate Internet Distance Estimation System
TLDR
This work proposes AIDE, a new accurate Internet distance estimation system based on matrix factorization, which is decentralized with robustness, and demonstrates that AIDE performs far more accurate estimation in short Internet latency.
DISCS: A Distributed Coordinate System Based on Robust Nonnegative Matrix Completion
TLDR
This paper presents a novel distributed network coordinate system (DISCS) that utilizes a limited set of distance measurements to achieve high-precision distance prediction at a fast convergence speed and finds that DISCS outperforms the state-of-the-art NCS schemes in terms of prediction precision and convergence speed.
Replacing Network Coordinate System with Internet Delay Matrix Service (IDMS): A Case Study in Chinese Internet
TLDR
It is shown that not all the hosts need an independent network coordinate (NC), and an Internet Delay Matrix Service (IDMS) for representing network distances without explicit measurements in Internet is presented, which is better than other existing approaches.
EigenSP: A More Accurate Shortest Path Distance Estimation on Large-Scale Networks
TLDR
A novel method EigenSP is presented, that estimates the shortest-path distance by using an adjacency matrix approximated by a few eigenvalues and eigenvectors, and empirical results suggest that Eigen SP estimates small distances better than the landmark-based methods.
TSMF: Network Latency Estimation using Matrix Factorization and Time Series Forecasting
TLDR
TSMF is devised, a novel NCS method based on an ensemble of Time-Series Forecasting and Matrix Factorization that outperforms Euclidean and MF-based NCS's with up to 6× less relative error in predicting end-to-end latencies.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 35 REFERENCES
Modeling distances in large-scale networks by matrix factorization
TLDR
The approach overcomes several limitations of previous coordinates-based mechanisms, which cannot model sub-optimal routing or asymmetric routing policies, and leads to more accurate, efficient and robust predictions of latencies in large-scale networks than previous approaches.
Vivaldi: a decentralized network coordinate system
TLDR
Vivaldi is a simple, light-weight algorithm that assigns synthetic coordinates to hosts such that the distance between the coordinates of two hosts accurately predicts the communication latency between the hosts.
Predicting Internet network distance with coordinates-based approaches
  • T. Ng, Hui Zhang
  • Computer Science
    Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies
  • 2002
TLDR
This work proposes using coordinates-based mechanisms in a peer-to-peer architecture to predict Internet network distance (i.e. round-trip propagation and transmission delay), and proposes the GNP approach, based on absolute coordinates computed from modeling the Internet as a geometric space.
PIC: practical Internet coordinates for distance estimation
TLDR
PIC is introduced, a practical coordinate-based mechanism to estimate Internet network distance that does not rely on infrastructure nodes and it can compute accurate coordinates even when some peers are malicious.
Practical, distributed network coordinates
TLDR
This paper evaluates Vivaldi through simulations of 750 hosts, with a matrix of inter-host latencies derived from measurements between 750 real Internet hosts, and finds synthetic coordinates that predict the measured latencies with a median relative error of 14 percent.
King: estimating latency between arbitrary internet end hosts
TLDR
The accuracy of King is significantly better than the accuracy of IDMaps, and that King tends to preserve order among its latency estimates, as well as a variety of measurement studies and applications that could benefit from the tool.
IDMaps: a global internet host distance estimation service
TLDR
It is shown that IDMaps can indeed provide useful distance estimations to applications such as nearest mirror selection, and is suggested a scalable Internet-wide architecture, called IDMaps, which measures and disseminates distance information on the global Internet.
Geometric Exploration of the Landmark Selection Problem
TLDR
This paper leverages work in coordinate systems to treat the problem of landmark selection geometrically, providing an opening for applying new algorithms and suggests that when employing small numbers of landmarks, such geometric algorithms provide good landmarks, while for largerNumbers of landmarks even faster methods based on random selection are equally effective.
Virtual landmarks for the internet
TLDR
This paper develops a faster method based on dimensionality reduction of the Lipschitz embedding that is reasonably accurate, even when applied to measurements spanning the Internet, and that it naturally leads to a scalable measurement strategy based on the notion of virtual landmarks.
Some findings on the network performance of broadband hosts
TLDR
An effort to directly measure P2P network performance from the vantage point of broadband-connected residential hosts and finds that the availability of last-hop bandwidth is more important than the traditional consideration of locality for overlay multicast over broadband hosts.
...
1
2
3
4
...