A Real Time Processing system for big data in astronomy: Applications to HERA

@article{Plante2021ART,
  title={A Real Time Processing system for big data in astronomy: Applications to HERA},
  author={Paul La Plante and Peter K. G. Williams and Matthew Kolopanis and Joshua S. Dillon and Adam P. Beardsley and Nicholas S. Kern and Michael J. Wilensky and Zaki S. Ali and Zara Abdurashidova and James E. Aguirre and Paul Alexander and Yanga Balfour and Gianni Bernardi and Tashalee S. Billings and Judd D. Bowman and Richard F. Bradley and Philip Bull and Jacob Burba and Steve H. Carey and Christopher L. Carilli and Carina Cheng and David R. DeBoer and Matt Dexter and Eloy de Lera Acedo and J. A. Ely and Aaron Ewall-Wice and Nicolas Fagnoni and Randall Fritz and Steven R. Furlanetto and Kingsley Gale-Sides and B. E. Glendenning and Deepthi Gorthi and Bradley Greig and Jaspar Grobbelaar and Ziyaad Halday and Bryna J. Hazelton and Jacqueline N. Hewitt and Jack Hickish and Daniel C. Jacobs and Austin Julius and Joshua Kerrigan and Piyanat Kittiwisit and Saul A. Kohn and Adam E. Lanman and Telalo Lekalake and David Lewis and Adrian Liu and David Macmahon and Lourence Malan and Cresshim Malgas and M. D. J. Maree and Zachary E. Martinot and Eunice Matsetela and Andrei Mesinger and Mathakane Molewa and Miguel F. Morales and Tshegofalang Mosiane and S. Murray and Abraham R. Neben and Bojan Nikoli{\'c} and Aaron Parsons and Robert Pascua and Nipanjana Patra and Samantha Pieterse and Jonathan C. Pober and Nima Razavi-Ghods and Jon Ringuette and James Robnett and Kathryn Rosie and Mario G Santos and Peter H Sims and Craig H. Smith and Angelo Syce and Nithyanandan Thyagarajan and Haoxuan Zheng},
  journal={Astron. Comput.},
  year={2021},
  volume={36},
  pages={100489}
}
1 Citations

Figures from this paper

DeepAdversaries: Examining the Robustness of Deep Learning Models for Galaxy Morphology Classification
TLDR
It is shown that training with domain adaptation improves model robustness and mitigates the effects of these perturbations, improving the classification accuracy by 23% on data with higher observational noise and increasing the latent space distance between the baseline and the incorrectly classi⬁ed one-pixel perturbed image, making the model more robust to inadvertent perturbation.

References

SHOWING 1-10 OF 14 REFERENCES
Absolute Calibration Strategies for the Hydrogen Epoch of Reionization Array and Their Impact on the 21 cm Power Spectrum
We discuss absolute calibration strategies for Phase I of the Hydrogen Epoch of Reionization Array (HERA), which aims to measure the cosmological 21 cm signal from the Epoch of Reionization (EoR).
Mitigating Internal Instrument Coupling for 21 cm Cosmology. II. A Method Demonstration with the Hydrogen Epoch of Reionization Array
We present a study of internal reflection and cross coupling systematics in Phase I of the Hydrogen Epoch of Reionization Array (HERA). In a companion paper, we outlined the mathematical formalism
Redundant-baseline calibration of the hydrogen epoch of reionization array
In 21-cm cosmology, precision calibration is key to the separation of the neutral hydrogen signal from very bright but spectrally smooth astrophysical foregrounds. The Hydrogen Epoch of Reionization
Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline
TLDR
The validation test suite uncovered several sources of scale-independent signal loss throughout the HERA Phase I software pipeline, whose amplitude is well-characterized and accounted for in the final estimates.
Unlocking the LOFAR LTA
TLDR
This paper enables a simple one-click-reduction of LOFAR observations into sky maps for any user of this archive, as part of the PROCESS 1 project, which aims to provide generalizable open source solutions for user friendly exascale data processing.
OPENING THE 21 cm EPOCH OF REIONIZATION WINDOW: MEASUREMENTS OF FOREGROUND ISOLATION WITH PAPER
We present new observations with the Precision Array for Probing the Epoch of Reionization with the aim of measuring the properties of foreground emission for 21 cm epoch of reionization (EoR)
The Hydrogen Epoch of Reionization Array (HERA)
  • D. DeBoer
  • Physics
    2016 International Conference on Electromagnetics in Advanced Applications (ICEAA)
  • 2016
The Hydrogen Epoch of Reionization Array (HERA http://reionization.org) is a staged experiment that uses the unique properties of the 21-cm line from neutral hydrogen to probe the Epoch of
Makeflow: a portable abstraction for data intensive computing on clusters, clouds, and grids
TLDR
This work introduces Makeflow, a simple system for expressing and running a data-intensive workflow across multiple execution engines without requiring changes to the application or workflow description, and introduces Workbench, a suite of benchmarks designed for analyzing common workflow patterns.
Biocompute: towards a collaborative workspace for data intensive bio-science
TLDR
The goals and interface to the Biocompute system, its architecture and performance, and the insights gained in its development are described.
Baseline-dependent averaging in radio interferometry
This paper presents a detailed analysis of the applicability and benefits of baseline dependent averaging (BDA) in modern radio interferometers and in particular the Square Kilometre Array (SKA). We
...
...