Intrusion Detection systems are now an essential component in the overall network. With the rapid advancement in the network technologies including higher bandwidths and ease of connectivity of wireless and hand held devices, the main focus of intrusion detection has shifted from simple signature matching approaches to detecting attacks based on analyzing contextual information which may be specific to individual networks and applications. As a result, anomaly and hybrid intrusion detection approaches have gained significance. The Denial of Service Attacks (DoS), Probe, User to Root (U2R) and Remote to Local (R2L) are some of the common attacks that affect network resources. Intrusion detection faces a number of challenges; an intrusion detection system must reliably detect malicious activities in a network and cope up with large amount of network traffic. In this paper, we address these two issues of Accuracy and Efficiency using Conditional Random Fields and Layered Approach. Finally we demonstrate that high attack detection accuracy can be achieved by using Memetic algorithm for feature selection with Layered Conditional Random Fields.