• Corpus ID: 2764668

TeraByte TokuSampleSort

@inproceedings{Kuszmaul2007TeraByteT,
  title={TeraByte TokuSampleSort},
  author={Bradley C. Kuszmaul},
  year={2007}
}
Using the tx2500 disk cluster at MIT Lincoln Laboraties, I so rted a terabyte (10 10 100-byte records) in 197s using an “Indy” sort, and in 297s using a “Daytona” sort. I sorted 264GB in one minut e using an “Indy” sort and 214GB in one minute using an “Daytona ” sort. The sort employed a parallel sample sort, and ran on 400 nodes, each containing a 6-node RAID, and 8GB of memory, all connected by infiniband. I employed TCP sockets to communica te between the nodes. I used a FUSE module for the… 
1 Citations

Figures from this paper

TritonSort: A Balanced Large-Scale Sorting System
We present TritonSort, a highly efficient, scalable sorting system. It is designed to process large datasets, and has been evaluated against as much as 100 TB of input data spread across 832 disks in

References

SHOWING 1-10 OF 11 REFERENCES
Sorting on a Cluster Attached to a Storage-Area Network
In November 2004, the SAN Cluster Sort program (SCS) set new records for the Indy versions of the Minute and TeraByte Sorts. SCS ran on a cluster of 40 dual-processor Itanium2 nodes on the show floor
A Minute with Nsort on a 32P NEC Windows Itanium2 Server
In March 2004, the Nsort program was able to sort 34 GB of data (340,000,000 100-byte records) in 58 seconds on a 32 processor Itanium® 2 NEC® Express5800/1320Xd running Microsoft® Windows® Server
An Experimental Analysis of Parallel Sorting Algorithms
TLDR
A methodology for predicting the performance of parallel algorithms on real parallel machines and selected the three most promising, Batcher's bitonic sort, a parallel radix sort, and a sample sort similar to Reif and Valiant's flashsort, and implemented them on the connection Machine model CM-2.
Ssh file system. http://fuse.sourceforge.net/sshfs. html
  • Ssh file system. http://fuse.sourceforge.net/sshfs. html
  • 2006
A minute of mainframe batch sorting on Windows
  • A minute of mainframe batch sorting on Windows
  • 2006
Filesystem in userspace (fuse). http://fuse.sourceforge. net
  • Filesystem in userspace (fuse). http://fuse.sourceforge. net
  • 2006
A minute of mainframe batch sorting on Windows. http://research.microsoft. com/barc/SortBenchmark/2006 NeoSortMinute.pdf
  • 2006
Sort benchmark home
  • page. http://research. microsoft.com/barc/SortBenchmark/,
  • 2006
Sorting on a cluster attached to a storagearea network. http://research.microsoft.com/barc/ SortBenchmark/2005 SCS Wyllie.pdf
  • 2005
Sort benchmark home page. http://research. microsoft.com/barc/SortBenchmark
  • Sort benchmark home page. http://research. microsoft.com/barc/SortBenchmark
  • 2006
...
...