Diffusion Adaptation over Networks

@article{Sayed2012DiffusionAO,
  title={Diffusion Adaptation over Networks},
  author={Ali H. Sayed},
  journal={ArXiv},
  year={2012},
  volume={abs/1205.4220}
}
  • A. Sayed
  • Published 18 May 2012
  • Computer Science
  • ArXiv

Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks

It is confirmed that under constant step-sizes, diffusion strategies allow information to diffuse more thoroughly through the network and this property has a favorable effect on the evolution of the network: diffusion networks are shown to converge faster and reach lower mean-square deviation than consensus networks, and their mean- square stability is insensitive to the choice of the combination weights.

Affine Combination of Diffusion Strategies Over Networks

This work proposes a combination framework that aggregates the operation of multiple diffusion strategies for enhanced performance by assigning a combination coefficient to each node, and using an adaptation mechanism to minimize the network error.

On the Influence of Informed Agents on Learning and Adaptation Over Networks

This work examines the performance of LMS diffusion strategies for distributed estimation over networks as a function of the proportion of informed agents and their distribution in space and shows that uninformed agents play an important role in determining the steady-state performance of the network and that it is preferable to keep some of the highly noisy or highly connected agents uninformed.

Adaptive Diffusion Schemes for Heterogeneous Networks

Several experiments involving networks with heterogeneous nodes show that the proposed decoupled adapt-then-combine approach with adaptive combiners outperforms other state-of-the-art techniques, becoming a competitive approach in these scenarios.

Zeroth-Order Diffusion Adaptation Over Networks

  • Jie ChenSijia LiuPin-Yu Chen
  • Computer Science
    2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2018
This work proposes the zeroth-order (ZO) diffusion strategy using randomized gradient estimates, and simulations are performed to examine properties of the algorithm and to compare it with its non-cooperative and stochastic gradient counterparts.

Adaptive Networks

  • A. Sayed
  • Computer Science
    Proceedings of the IEEE
  • 2014
Under reasonable technical conditions on the data, the adaptive networks are shown to be mean square stable in the slow adaptation regime, and their mean square error performance and convergence rate are characterized in terms of the network topology and data statistical moments.

Distributed Estimation for Adaptive Networks Based on Serial-Inspired Diffusion

This work develops a serial-inspired approach based on message-passing strategies that provides a significant improvement in performance over prior art and makes use of the most up-to-date information in the graph in combination with the diffusion approach to offer improved performance.

Adaptive clustering for multitask diffusion networks

An unsupervised strategy that allows each node to continuously select the neighboring nodes with which it should exchange information to improve its estimation accuracy is derived.

On the Learning Behavior of Adaptive Networks

The analysis reveals how combination policies influence the learning process of networked agents, and how these policies can steer the convergence point towards any of many possible Pareto optimal solutions and suggests design procedures that can optimize performance by adjusting the relevant topology parameters.

Convex Combination of Diffusion Strategies Over Distributed Networks

  • Danqi JinJie ChenJingdong Chen
  • Computer Science
    2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
  • 2018
Inspired by the convex combination of adaptive filters, this paper proposes to benefit the performance of two distinct strategies by appropriately combining their fusion coefficients, with both static and dynamic fusion components.
...

References

SHOWING 1-10 OF 101 REFERENCES

Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks

An adaptive diffusion mechanism to optimize global cost functions in a distributed manner over a network of nodes, which endow networks with adaptation abilities that enable the individual nodes to continue learning even when the cost function changes with time.

Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks

It is confirmed that under constant step-sizes, diffusion strategies allow information to diffuse more thoroughly through the network and this property has a favorable effect on the evolution of the network: diffusion networks are shown to converge faster and reach lower mean-square deviation than consensus networks, and their mean- square stability is insensitive to the choice of the combination weights.

Diffusion Adaptation Over Networks Under Imperfect Information Exchange and Non-Stationary Data

This paper investigates the mean-square performance of general adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges, quantization errors, and model non-stationarities and reveals that link noise over the regression data modifies the dynamics of the network evolution in a distinct way, and leads to biased estimates in steady-state.

Diffusion networks outperform consensus networks

Diffusion strategies allow information to diffuse more thoroughly through the network, and this property has a favorable effect on the evolution of the network: diffusion networks reach lower mean-square deviation than consensus networks, and their mean- square stability is insensitive to the choice of the combination weights.

Combination weights for diffusion strategies with imperfect information exchange

This paper investigates the mean-square performance of adaptive diffusion algorithms in the presence of various sources of imperfect information exchanges and quantization errors, and reveals that link noise over the regression data modifies the dynamics of the network evolution, and leads to biased estimates in steady-state.

Spatio-Temporal Diffusion Strategies for Estimation and Detection Over Networks

The results indicate that whether temporal processing is performed before or after adaptation, the strategy that performs adaptation before spatial cooperation leads to smaller mean-square error.

Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis

Closed-form expressions that describe the network performance in terms of mean-square error quantities are derived and the resulting algorithm is distributed, cooperative and able to respond in real time to changes in the environment.

Spatio-temporal diffusion mechanisms for adaptation over networks

This work develops diffusion algorithms for adaptation over networks that endow nodes with both cooperation abilities and temporal processing abilities and indicates that the version that performs adaptation prior to the steps of spatial cooperation and temporalprocessing leads to best performance.

Optimal combination rules for adaptation and learning over networks

  • Sheng-Yuan TuA. Sayed
  • Computer Science
    2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)
  • 2011
This work considers the problem of optimal selection of the combination weights and motivates one combination rule, related to the inverse of the noise variances, which is shown to be effective in simulations.

Diffusion recursive least-squares for distributed estimation over adaptive networks

This work proposes a diffusion recursive least-squares algorithm where nodes need to communicate only with their closest neighbors and requires no transmission or inversion of matrices, therefore saving in communications and complexity.
...