The charge state of individually addressable impurities in semiconductor material was manipulated with a scanning tunneling microscope. The manipulation was fully controlled by the position of the tip and the voltage applied between tip and sample. The experiments were performed at low temperature on the (110) surface of silicon doped GaAs. Silicon donors up to 1 nm below the surface can be reversibly switched between their neutral and ionized state by the local potential induced by the tip. By using ultrasharp tips, the switching process occurs close enough to the impurity to be observed as a sharp circular feature surrounding the donor. By utilizing the controlled manipulation, we were able to map the Coulomb potential of a single donor at the semiconductor-vacuum interface.