Scaling big data mining infrastructure: the twitter experience

@article{Lin2012ScalingBD,
  title={Scaling big data mining infrastructure: the twitter experience},
  author={Jimmy J. Lin and Dmitriy V. Ryaboy},
  journal={SIGKDD Explorations},
  year={2012},
  volume={14},
  pages={6-19}
}
The analytics platform at Twitter has experienced tremendous growth over the past few years in terms of size, complexity, number of users, and variety of use cases. In this paper, we discuss the evolution of our infrastructure and the development of capabilities for data mining on "big data". One important lesson is that successful big data mining in practice is about much more than what most academics would consider data mining: life "in the trenches" is occupied by much preparatory work that… CONTINUE READING

Figures, Tables, and Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 91 CITATIONS

Public auditing, analytics, and big data in the modern economy

VIEW 10 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Propeller: A Scalable Real-Time File-Search Service in Distributed Systems

  • 2014 IEEE 34th International Conference on Distributed Computing Systems
  • 2014
VIEW 6 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

VSFS: A Searchable Distributed File System

  • 2014 9th Parallel Data Storage Workshop
  • 2014
VIEW 4 EXCERPTS
CITES BACKGROUND & METHODS
HIGHLY INFLUENCED

Big data analytics for video surveillance

  • Multimedia Tools and Applications
  • 2019
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2012
2019

CITATION STATISTICS

  • 7 Highly Influenced Citations

  • Averaged 11 Citations per year from 2017 through 2019

References

Publications referenced by this paper.