Deepak Rajan

Learn More
Originally, MapReduce implementations such as Hadoop employed First In First Out (fifo) scheduling, but such simple schemes cause job starvation. The Hadoop Fair Scheduler (hfs) is a slot-based MapReduce scheme designed to ensure a degree of fairness among the jobs, by guaranteeing each job at least some minimum number of allocated slots. Our prime(More)
This paper describes the SODA scheduler for System S , a highly scalable distributed stream processing system. Unlike traditional batch applications, streaming applications are open-ended. The system cannot typically delay the processing of the data. The scheduler must be able to shift resource allocation dynamically in response to changes to resource(More)
We study the polyhedra of splittable and unsplittable single arc–set relaxations of multicommodity flow capacitated network design problems. We investigate the optimization problems over these sets and the separation and lifting problems of valid inequalities for them. In particular, we give a linear–time separation algorithm for the residual capacity(More)
LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be(More)
In this paper, we describe an optimization scheme for fusing compile-time operators into reasonably-sized run-time software units called processing elements (PEs). Such PEs are the basic deployable units in System S, a highly scalable distributed stream processing middleware system. Finding a high quality fusion significantly benefits the performance of(More)
Virtualized cloud systems are prone to performance anomalies due to various reasons such as resource contentions, software bugs, and hardware failures. In this paper, we present a novel Predictive Performance Anomaly Prevention (PREPARE) system that provides automatic performance anomaly prevention for virtualized cloud computing infrastructures. PREPARE(More)
A network is said to be survivable if it has sufficient capacity for rerouting all of its flow under the failure of any one of its edges. Here we present a polyhedral approach for designing survivable networks. We describe a mixed–integer programming model, in which sufficient slack is explicitly introduced on the directed cycles of the network while flow(More)
We give strong formulations of ramping constraints — used to model the maximum change in production level for a generator or machine from one time period to the next — and production limits. For the two-period case, we give the first complete description of the convex hull of the feasible solutions. The two-period inequalities can be readily used to(More)
Stochastic mixed-integer programs (SMIPs) deal with optimization under uncertainty at many levels of the decision-making process. When solved as extensive formulation mixed-integer programs, problem instances can exceed available memory on a single workstation. To overcome this limitation, we present PIPS-SBB: an exact distributed-memory parallel stochastic(More)