The plasmon resonance of metal nanoparticles shifts upon refractive index changes of the surrounding medium through the binding of analytes. The use of this principle allows one to build ultra-small plasmon sensors that can detect analytes (e.g., biomolecules) in volumes down to attoliters. We use simulations based on the boundary element method to determine the sensitivity of gold nanorods of various aspect ratios for plasmonic sensors and find values between 3 and 4 to be optimal. Experiments on single particles confirm these theoretical results. We are able to explain the optimum by showing a corresponding maximum for the quality factor of the plasmon resonance.