................................................................................................................................ 1 Background ........................................................................................................................... 1 Methods................................................................................................................................. 4 Developing Key Foundational Ideas and Knowledge Specifications ............................... 4 From Knowledge to Item Specifications .......................................................................... 7 Technical Quality of Assessment Items ............................................................................ 8 Evaluating the Evidence of Technical Quality ............................................................... 10 Description of the Rational Number Video Game (PuppetMan) Task ........................... 10 The Sample ..................................................................................................................... 12 Determining the Technical Quality of the Game and Other Measures ........................... 14 Results ................................................................................................................................. 16 Content Coverage............................................................................................................ 16 Relationship between Tasks Requiring the Same Content Knowledge .......................... 22 Relationship between Items on a Particular Test Form (Interitem Reliability) .............. 26 Item Test-Retest Reliability ............................................................................................ 31 Item Test-Retest Effects .................................................................................................. 32 Form Test-Retest Reliability ........................................................................................... 32 Conclusions ......................................................................................................................... 33 References ........................................................................................................................... 36 Appendix A Knowledge Specifications .................................................................................. 39 Appendix B PuppetMan Screen Shots from the First Version Tested with Students ............. 45