Learn More
Portals Direct I/O ("PDIO") is a specialpurpose middleware infrastructure for writing data from compute processor memory on Portals-enabled compute nodes to remote agents anywhere on the WAN in realtime. The prototype implementation provided a means for aggregation of outgoing data through multiple loadbalanced routing daemons, end-to-end parallel data(More)
Growing compute capacity coupled with advances in parallel filesystem performance and stability mean that HPC users will inevitably create and store larger datasets. If data residing on parallel filesystems is not efficiently offloaded to archival storage, disruptions in the compute cycle will occur. Hierarchical storage caches are a vital aspect of the HPC(More)
Traditionally, Hadoop is run on parallel machines with modest processing capabilities and large amounts of disk. A large shared-memory system may not, at first blush, seem a likely host for a Hadoop cluster, but when researchers asked to run Hadoop on our 16 TB shared memory system, Blacklight, we set out to determine how effectively Hadoop could work in(More)
  • 1