In-situ I/O processing: a case for location flexibility

@inproceedings{Zheng2011InsituIP,
  title={In-situ I/O processing: a case for location flexibility},
  author={Fang Zheng and Hasan Abbasi and Jianting Cao and Jai Dayal and Karsten Schwan and Matthew Wolf and Scott Klasky and Norbert Podhorszki},
  booktitle={PDSW '11},
  year={2011}
}
Increasingly severe I/O bottlenecks on High-End Computing machines are prompting scientists to process output data during simulation time, "in-situ", and before placing data on disks. This paper argues for flexibility in the implementation of such in-situ data analytics, using measurements and a performance model that demonstrate the potential advantages and limitations of performing analytics at different levels of the I/O hierarchy, including on a machine's compute nodes vs. on separate… CONTINUE READING