Genetic expression of a trait is complicated and it is usually associated with many genes including their interactions (epistasis) and genotype-by-environment interactions. Genetic mapping currently focuses primarily on additive models or marginal genetic effects due to the complexity of epistatic effects. Thus, there exists a need to appropriately identify favorable epistatic effects for important biological traits. Several multifactor dimensionality reduction (MDR) based methods are important resources to identify high-order gene–gene interactions. These methods are mainly focused on human genetic studies. Many traits in plant systems are not only quantitatively inherited but also are often measured in repeated field plots under multiple environments. In this study, we proposed a mixed model based MDR approach, which is suitable for inclusion of various fixed and random effects. This approach was used to analyze a cotton data set that included eight agronomic and fiber traits and 20 DNA markers. The results revealed high order epistatic effects were detected for most of these traits using this modified MDR approach.