Because of the non-contact and substantivity, recognition from images have become a hotspot in somatic technology. Especially, recognition with non-markers lets human body become remote controller without any accessories and attracts widespread attention. This paper chose CAM-Shift algorithm to track bare hands for its calculate speed and stability. This article introduced depth information to traditional CAM-Shift and developed and extended the algorithm in foreground segmentation, and auto-initialization. This paper mainly segmented depth map to get the mask off code of the foreground, and made use of the 3D information of the depth maps to initialize the searching window. Then, the paper built up Interaction System of Treadmill Games based on depth maps and CAM-Shift. The system realized the man-machine interaction with non-contact and no-markers by getting player's depth maps and color images with a attachment of Xbox — Kinect from Microsoft, 3D tracking the player's bare hands and analyzing and recognizing the movements of the hands. Then the experiments testified this system satisfies the real time request, and the reaction is acute and correct.