Scratch input

Short description: Acoustic-based human-computer interface

Scratch input using geophones (contact microphones) bonded to each fingernail as a musical instrument. The awful sounds generated by running fingernails down the chalkboard are separately pitch-corrected, resulting in a pleasant-sounding musical instrument.

In computing, scratch input is an acoustic-based method of Human-Computer Interaction (HCI) that takes advantage of the characteristic sound produced when a finger nail or other object is dragged over a surface, such as a table or wall. The technique is not limited to fingers; a stick or writing implements (e.g. chalk, or a pen) can also be used. The sound is often inaudible to the naked ear (i.e., silent). However, specialized microphones can digitize the sounds for interactive purposes. Scratch input was invented by Mann et al. in 2007, though the term was first used by Chris Harrison et al.

History

A natural interface for musical expression operating on scratch input principles was first published and presented in June 2007. Later that year, it was extended to an implementation on a smartphone and also a wearable computer system.

In 2008, the Scratch Input project demonstrated a mobile device input system utilizing scratch input, simultaneously popularizing the term. This system captured audio transmitted through a surface on which a mobile phone was placed, enabling the entire surface to be used as an input device.

Uses

Scratch input is an enabling input technique that is used in multitude of applications. The earliest application was a highly expressive musical instrument (Mann et al.) for use with mobile devices on natural objects, surfaces, or the like, as a non-synthesizing (i.e. idiophonic) musical instrument. Harrison et al. proposed it to create large, ad hoc gestural input areas when mobile devices are rested on tables.

Scratching a stick on bricks and mortar, with vision-assisted gesture recognition. This gives better results than scratch input alone. (Public performance in Toronto, Ontario; image from the 2007 paper.)

Scratch input with ice skates is called a pagophone.

Commercial potential

Microsoft has expressed interest in Scratch Input.

With this we can start to think of every flat surface as an potential input area. If mass produced this sensor could cost less than a dollar....Despite the limitations, the technology holds enough promise to make it into the hands of consumers. It is exciting because it is so low cost. This idea has the potential to go beyond just a research project.

– Daniel Wigdor, User experience architect at Microsoft and curator of the emerging technology demos at SIGGRAPH

See also

• Scratch Input with ice skates
• Vision-assisted Scratch Input
• Scratch Input explanation and demonstration