Eye-Tracking Metrics in Software Engineering
Designing and understanding of diagrammatic representations is a critical issue for the success of software projects because diagrams in this field provide a collection of related information with various perceptual signs and they help software engineers to understand operational systems at different levels of information system development process. Entity relationship diagram (ERD) is one of the main diagrammatic representations of a conceptual data model that reflects users' data requirements in a database system. In today's business environment, the business model is in a constant change which creates highly dynamic data requirements which also requires additional processes like modifications of ERD. However, in the literature there are not many measures to better understand the behaviors of software engineers during designing and understanding these representations. Hence, the main motivation of this study is to develop measures to better understand performance of software engineers during their understanding process of ERD. Accordingly, this study proposes two measures for ERD defect detection process. The defect detection difficulty level (DF) measures how difficult a defect to be detected according to the other defects for a group of software engineers. Defect detection performance (PP) measure is also proposed to understand the performance of a software engineer during the defect detection process. The results of this study are validated through the eye tracker data collected during the defect detection process of participants. Additionally, a relationship between the defect detection performance (PP) of a software engineer and his/her search patterns within an ERD is analyzed. Second experiment with five participants is also conducted to show the correlation between the proposed metric results and eye tracker data. The results of experiment-2 also found to be similar for DF and PP values. The results of this study are expected to provide insights to the researchers, software companies, and to the educators to improve ERD reasoning process. Through these measures several design guidelines can be developed for better graphical representations and modeling of the information which would improve quality of these diagrams. Moreover, some reviewing instructions can be developed for the software engineers to improve their reviewing process in ERD. These guidelines in turn will provide some tools for the educators to improve design and review skills of future software engineers.