We consider spin glass models in which the number of spin components m is infinite. In the formulation of the problem appropriate for numerical calculations proposed by several authors, we show that the order parameter defined by the long-distance limit of the correlation functions is actually zero and there is only "quasi-long-range order" below the transition temperature. Nonetheless, there can be a finite temperature phase transition where the decay of correlations changes from exponential to power law. We also show that the spin glass transition temperature is zero in three dimensions so power-law behavior only occurs at T=0 in this case. We also argue that the order of limits, m-->infinity and N-->infinity is important, where N is the number of spins.