Video scene classification technology is becoming more important with development of multi-media system. In order to classify the video scene, the models to identify the video scene are needed. In the case of constructing the video scene classification system for TV programs, it is difficult to make the models because the T V programs have many kinds of scene. For above mentioned problem, a new type approach for video scene classification based on typical scene model dat,abase is proposed. In this approach, the classification of video scene is realized using similarity retrieval from the image database. The typical scene model database is consist of collection of the features of image of typical scene. The advantage of this approach is that this approach can classify the scene of image without recognition of object. Consequently, the user do not have to prepare the knowledge for recognition of objects and scenes. The user has only to collect the image of typical scene. In prototype system, the similarity of image is evaluated using the feature of color information and segment information of image.