In this paper, we propose a novel pictograph detection scheme by removing miss matching points. In our previous work, we developed a robot navigation which uses only one CCD camera equipped on the robot and estimates localize self-position by pictographs on a general environment. However, our system sometimes misjudges feature points on the background as feature points on pictographs. Threfore, our proposed scheme removes these misjudge points by making color histograms. Misjudge points are distinguished from their similarity, and are removed. In matching experiment, we verified correcting misjudge points and the error of the detected position of the pictograph in the image. As a result, in every kind of pictographs which were used for the experiment, we verified the effectiveness of our proposed methods.