Sketch Misrecognition Correction Based on Eye Gaze Monitoring

Abstract/Technology Overview

A novel gaze-based error recognition system. Prediction of user intention to correct user drawn sketch misrecognition through a computer-based intelligent user interface. The present invention relates to a gaze-based error recognition detection system that is intended to predict the intention of the user to correct user drawn sketch misrecognition through a multimodal computer-based intelligent user interface.

Technology Features, Specifications and Advantages

Sketch recognition is the segmentation and interpretation of sketches by the computer in a human-like manner. Unlike plain pen-based interfaces, which see pen input merely as a collection of ink coordinates, sketch-based 'intelligent interfaces are able to interpret hand-drawn sketches. A sketch-based intelligent interface segments user sketches into meaningful pieces and labels each segment as an object. Sketch-based interfaces employ a variety of feedback strategies to indicate recognition results. The most common strategy is replacing sketch fragments with a recognized version of the intended object.

Sketch misrecognition is the incorrect labelling of a sketch segment, such as labelling the capacitor element on the circuit diagram sketch as a battery. In current sketch recognition systems, the user generally handles these mis-recognitions by erasing the misrecognition feedback and remaking the sketch or taking any other step to acquire correct sketch recognition feedback. These extra steps cause waste of time and impact efficiency. According to the present invention, a set of gaze-based features, which were designed to capture qualitative characteristics of users' eye gaze behaviour are used. The eye gaze-based error prediction system of the present invention provides an interface with sketch recognition technology to facilitate natural and efficient interaction through eye gaze behaviour of the user.

The invention, therefore, relates to a method of and system and device to predict the intention of the user to correctly sketch mis-recognitions by a set of gaze-based features designed to capture qualitative characteristics of eye gaze behaviour of the user.

Potential Applications

Sketch-based intelligent systems especially for educational and design purpose. 

The Global Market of Capacitive Stylus is growing rapidly mainly due to the rising market penetration of smart phones. According to a recent study report published by the Market Research Future, The global market of Capacitive Stylus is booming and expected to gain prominence over the forecast period. The global Capacitive Stylus Market is anticipated to accrete pervasively by 2023, with a whooping CAGR between 2018 and 2023.

Customer Benefit

• Correction of errors with at least 85-90% prediction accuracy
• Ability toread users intention enabling one-step correction 

Contact Person

Pelin Gundogdu


Koç University

Technology Category
  • Electronics
  • Infocomm
  • Artificial Intelligence
  • Educational Technology
Technology Status
  • Available for Licensing
Technology Readiness Level
  • TRL 3

gaze based error detection, machine learning, eye tracker, Misrecognition