We determine the capacity of a discrete memoryless communication channel with an energy harvesting transmitter and its battery state information available at the transmitter and the receiver. This capacity is an upper bound for the problem where side information is available only at the transmitter. Since channel output feedback does not increase the capacity in this case, we equivalently study the resulting finite-state Markov channel with feedback. We express the capacity in terms of directed information. Additionally, we provide sufficient conditions under which the capacity expression is further simplified to include the stationary distribution of the battery state. We also obtain a single-letter expression for the capacity with battery state information at both sides and an infinite-sized battery. Lastly, we consider achievable schemes when side information is available only at the transmitter for the case of an arbitrary finite-sized battery. We numerically evaluate the capacity and achievable rates with and without receiver side information.