We introduce a consistent matrix formalism for the characterization of partial polarization and coherence of random, nonstationary electromagnetic beams in time and frequency domains. We derive the temporal and spectral degrees of polarization and the Stokes parameters in terms of the time-domain and frequency-domain polarization matrices. The connections between temporal polarization and spectral coherence on the one hand and spectral polarization and temporal coherence on the other hand are discussed. Additionally, we establish equivalence theorems for fields with different temporal coherence properties to have the same spectral polarization states and for fields with different spectral coherence properties to possess identical temporal polarization. The theory is illustrated by analyzing specific examples of time-domain and frequency-domain electromagnetic Gaussian Schell-model pulsed beams.