Corpus ID: 54448088

K-Pg: Shared State in Differential Dataflows

@article{McSherry2018KPgSS,
  title={K-Pg: Shared State in Differential Dataflows},
  author={F. McSherry and Andrea Lattuada and M. Schwarzkopf},
  journal={ArXiv},
  year={2018},
  volume={abs/1812.02639}
}
Many of the most popular scalable data-processing frameworks are fundamentally limited in the generality of computations they can express and efficiently execute. In particular, we observe that systems' abstractions limit their ability to share and reuse indexed state within and across computations. These limitations result in an inability to express and efficiently implement algorithms in domains where the scales of data call for them most. In this paper, we present the design and… Expand
1 Citations
Online Analysis of Distributed Dataflows with Timely Dataflow
  • PDF

References

SHOWING 1-10 OF 32 REFERENCES
Naiad: a timely dataflow system
  • 652
  • PDF
Differential Dataflow
  • 125
  • PDF
Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing
  • 3,371
  • Highly Influential
  • PDF
TelegraphCQ: continuous dataflow processing
  • 756
Broom: Sweeping Out Garbage Collection from Big Data Systems
  • 79
  • PDF
Dryad: distributed data-parallel programs from sequential building blocks
  • 2,724
  • PDF
Transaction Repair: Full Serializability Without Locks
  • 7
  • PDF
DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language
  • 818
  • PDF
CIEL: A Universal Execution Engine for Distributed Data-Flow Computing
  • 271
  • PDF
...
1
2
3
4
...