Although several methods for action recognition have been proposed in the literature, many of them have limitations in terms of applicability in real-life situations. Despite satisfactory accuracy rates achieved by a number of methods, an effective action recognition system requires workability in real time. However, this feature usually comes along with certain loss in accuracy. In this paper, we present a real-time action recognition method that achieves state-of-the-art accuracy. By accumulating shape information over a sliding window on the video frames, the method extracts and processes silhouettes with little computational effort. Simple descriptors are computed over the shapes and applied on a fast configuration of classifiers. Experiments are conducted on three public data sets and the results demonstrate the effectiveness of the method in terms of accuracy and speed.