The main objective of this project is to develop a technique to classify the ripeness of bananas into 3 categories, which is unripe, ripe and overripe systematically based on their histogram RGB value components. This system involved the process of collecting samples with different level of ripeness, image processing and image classification by using artificial neural network. Collecting bananas sample is done by using Microsoft NX6000 webcam with 2 mega pixels. 32 samples were used as training samples for artificial neural network. In order to see whether the method mention above can classify the image correctly, another 28 images was used as a testing. From the result obtained, it was shown that the artificial neural network can generally classify the ripeness of bananas. This is because it can classify up to 25 samples correctly out of 28 samples. Developing a program totally by using Matlab version 7.0 can help classification process successfully.