A linear model of recurrent neural networks, called the Elman networks, is combined with the simple nonlinear modulo (mod) operation on its linear activated function so as to generate chaos purposely. Conditions on the weight matrix are obtained, under which the generated chaos satisfies the mathematical definition of chaos in the sense of T.Y. Li and J.A. Yorke (1975). Some simple and representative weight matrices are constructed for designing such Elman networks that can generate Li-Yorke chaos. Several numerical simulations are shown to verify and visualize the design.