Visual analysis of human motion in video sequences has attached more and more attention from computer visions in recent years. In order to detect human motion in Intelligent Security Monitoring System, moving body is detected and the boundary is extracted. According to the distance between contour points and the centroid, an exclusive 2-D (dimension) matrix is formed. In order to reduce computational cost affine transformation is proposed to normalize the matrix. Next the normalized matrix compares with the sequence which based formerly. The result is a vector, and then the standard deviation of the vectors is computed. Finally, Hidden Markov Models is used for human posture modeling and activity matching to recognize the human motion. Experiment results have shown that this method gives stable performances and good robustness.