Learn More
In this paper, we describe CogSketch, an open-domain sketch understanding system built on the nuSketch architecture. CogSketch captures the multi-modal, unconstrained nature of sketching by focusing on reasoning over recognition. We describe this approach, as well as two application domains for CogSketch: cognitive modeling, and education.
Sketching is a powerful means of working out and communicating ideas. Sketch understanding involves a combination of visual, spatial, and conceptual knowledge and reasoning, which makes it both challenging to model and potentially illuminating for cognitive science. This paper describes CogSketch, an ongoing effort of the NSF-funded Spatial Intelligence and(More)
This paper shows how a software toolkit enables graphic designers to make camera-based interactive environments in a short period of time without requiring experience in user interface design or machine vision. The Attentive Interaction Design Toolkit, a vision-based input toolkit, gives users an analysis of faces found in a given image stream, including(More)
This paper shows how a software toolkit can allow graphic designers to make camera-based interactive environments in a short period of time without experience in user interface design or machine vision. The Attention Meter, a vision-based input toolkit, gives users an analysis of faces found in a given image stream, including facial expression, body motion,(More)
Designers often use a series of sketches to explain how their design goes through different states or modes to achieve its intended function. Learning how to create such explanations turns out to be a difficult problem for engineering students. An automated “crash test dummy” to let students practice explanations would be desirable. This paper describes how(More)
Sketching is a powerful modality for thinking through, and communicating about, mechanical designs. Qualitative mechanics reasoning has been applied to sketched input before but not without succumbing to limitations of domainbased recognition or requiring complex annotation and additional explicit knowledge from the user. This paper presents solutions to(More)
Spatial reasoning plays a critical role in STEM problem solving. Physics assessments, for example, are rich in diagrams and pictures, which help people understand concrete physical scenarios and abstract aspects of physical reasoning. In this paper we describe a system that analyzes sketched diagrams to solve qualitative physics problems from a popular(More)
CogSketch is an open-domain sketch understanding system. CogSketch takes a unique approach to sketch understanding that focuses less on low-level recognition and more on highlevel reasoning with sketches. In addition to demonstrating the basic system, we will showcase applications to cognitive simulation and education. Overview of CogSketch CogSketch is an(More)
We have become accustomed to a digital desktop that is customizable and anticipates our needs. What if our physical work surfaces could interact with us in the same way? We have recently completed a working prototype of a work surface that automatically adjusts to individual users and anticipates their needs. The work surface is a large sink and counter(More)
Constructing models of dynamic systems is an important skill in both mathematics and science instruction. However, it has proved difficult to teach. Dragoon is an intelligent tutoring system intended to quickly and effectively teach this important skill. This paper describes Dragoon and an evaluation of it. The evaluation randomly assigned students in a(More)