Cytochrome c oxidase is the terminal enzyme in the electron transfer chain of essentially all organisms that utilize oxygen to generate energy. It reduces oxygen to water and harnesses the energy to pump protons across the mitochondrial membrane in eukaryotes and the plasma membrane in prokaryotes. The mechanism by which proton pumping is coupled to the oxygen reduction reaction remains unresolved, owing to the difficulty of visualizing proton movement within the massive membrane-associated protein matrix. Here, with a novel hydrogen/deuterium exchange resonance Raman spectroscopy method, we have identified two critical elements of the proton pump: a proton loading site near the propionate groups of heme a, which is capable of transiently storing protons uploaded from the negative-side of the membrane prior to their release into the positive side of the membrane and a conformational gate that controls proton translocation in response to the change in the redox state of heme a. These findings form the basis for a postulated molecular model describing a detailed mechanism by which unidirectional proton translocation is coupled to electron transfer from heme a to heme a 3, associated with the oxygen chemistry occurring in the heme a 3 site, during enzymatic turnover.