View source for Sonic interaction design

'''Sonic interaction design''' is the study and exploitation of [[Physics:Sound|sound]] as one of the principal channels conveying [[Information|information]], meaning, and aesthetic/emotional qualities in interactive contexts.<ref>Davide Rocchesso, Stefania Serafin, Frauke Behrendt, Nicola Bernardini, Roberto Bresin, Gerhard Eckel, Karmen Franinović, Thomas Hermann, Sandra Pauletto, Patrick Susini, and Yon Visell, (2008). ''Sonic interaction design: sound, information and experience''. In: CHI '08 Extended Abstracts on Human Factors in Computing Systems (Florence, Italy, April 05 – 10, 2008). CHI '08. ACM, New York, NY, 3969–3972. {{doi|10.1145/1358628.1358969}}</ref><ref>Davide Rocchesso and Stefania Serafin, (2009). "Sonic Interaction Design". Editorial of Special Issue. ''International Journal of Human–Computer Studies'' 67(11) (Nov. 2009):&nbsp;905–906. {{doi|10.1016/j.ijhcs.2009.09.009}}</ref> Sonic interaction design is at the intersection of [[Interaction design|interaction design]] and [[Sound and music computing|sound and music computing]]. If [[Interaction design|interaction design]] is about designing objects people interact with, and such interactions are facilitated by computational means, in sonic interaction design, sound is mediating interaction either as a display of processes or as an input medium.
== Research areas ==

=== Perceptual, cognitive, and emotional study of sonic interactions ===
[[Research]] in this area focuses on experimental scientific findings about human sound reception in [[Interactivity|interactive]] contexts.<ref>Guillaume Lemaitre, Olivier Houix, Yon Visell, Karmen Franinović, Nicolas Misdariis, and Patrick Susini, (2009). "Toward the design and evaluation of continuous sound in tangible interfaces: The Spinotron". ''International Journal of Human–Computer Studies'' 67(11) (Nov. 2009):&nbsp;976–993. {{doi|10.1016/j.ijhcs.2009.07.002}}</ref>

During closed-loop interactions, the users manipulate an interface that produces sound, and the sonic feedback affects in turn the users’ manipulation. In other words, there is a  tight coupling between auditory [[Perception|perception]] and action.<ref>Salvatore M. Aglioti and Mariella Pazzaglia (2010). "Representing actions through their sound". "Experimental Brain Research" 206(2):&nbsp;141–151. {{doi|10.1007/s00221-010-2344-x}}</ref> Listening to sounds might not only activate a representation of how the  sound was made: it might also prepare the listener to react to the sound. Cognitive representations of sounds might be associated with action-planning  schemas, and sounds can also unconsciously cue a further reaction on the part  of the listener.<ref>Marzia De Lucia, Christian Camen, Stephanie Clarke, and Micah M. Murray (2009). "The role of actions in auditory ob ject discrimination". "Neuroimage" 48(2):&nbsp;475–485. {{doi|10.1016/j.neuroimage.2009.06.041}}</ref>

Sonic interactions have the potential to influence the users’ [[Philosophy:Emotion|emotion]]s: the quality of the sounds affects the pleasantness  of the interaction, and the difficulty of the manipulation influences whether the user feels in control or not.<ref>Guillaume Lemaitre, Olivier Houix, Karmen Franinović, Yon Visell, and Patrick Susini (2009). "The Flops glass: a device to study the emotional reactions arising from sonic interactions". In Proc. Sound and Music Computing Conference, Porto,  Portugal.  Available: [http://smc2009.smcnetwork.org/programme/pdfs/281.pdf] {{Webarchive|url=https://web.archive.org/web/20120225224405/http://smc2009.smcnetwork.org/programme/pdfs/281.pdf|date=2012-02-25}}</ref>

=== Product sound design ===
Product design in the context of sonic interaction design is dealing with methods and experiences for designing interactive products having a salient sonic behaviour. Products, in this context, are either tangible and functional objects that are designed to be manipulated,<ref>Daniel Hug (2008). ''Genie in a Bottle: Object–Sound Reconfigurations for Interactive Commodities''. In: Proceedings of Audiomostly 2008, 3rd Conference on Interaction With Sound (2008). Available: [http://www.citeulike.org/group/10826/article/7009573 online]</ref><ref>Karmen Franinović, Daniel Hug and Yon Visell (June 2007). ''Sound Embodied: Explorations of Sonic Interaction Design for Everyday Objects in a Workshop Setting''.  In: Proceedings of the 13th International Conference on Auditory Display, Montréal, Canada, June 26 – 29, 2007,  pp. 334–341. Available:  [http://www.icad.org/Proceedings/2007/FraninovicHug2007.pdf online]</ref> or usable simulations of such objects as in [[Engineering:Virtual prototyping|virtual prototyping]]. Research and development in this area relies on studies from other disciplines, such as:
* product [[Engineering:Sound quality|sound quality]];<ref>Richard H. Lyon, (2003). "Product sound quality-from perception to design". ''Sound and Vibration'' 37(3):&nbsp;18–22. Available: [http://www.sandv.com/downloads/0303lyon.pdf online]</ref>
* [[Physics:Acoustic ecology|acoustic ecology]], ''i.e.'' the relationship, mediated through sound, between living beings and their [[Earth:Ecosystem|environment]];
* film sound;<ref>See [http://filmsound.org/ filmsound.org: Learning Space dedicated to the Art of Film Sound Design]</ref>
* [[Engineering:Video game|computer and video game]] sound;<ref>See [http://www.gamesound.org/ About gamesound.org]</ref>
* sound culture, ''i.e.'' the study of how the production and consumption of sound have changed throughout history and within different societies.

In design research for sonic products a set of practices have been inherited from a variety of fields. Such practices have been tested in contexts where research and pedagogy naturally intermix. Among these practices it suffices to mention:
* [[Bodystorming|bodystorming]], especially when combined with vocal sketching,<ref>Inger Ekman and Michal Rinott, (2010). ''Using vocal sketching for designing sonic interactions'', Aarhus, Denmark: Designing Interactive Systems archive,
Proceedings of the 8th ACM Conference on Designing Interactive Systems, {{ISBN|978-1-4503-0103-9}}. Available: [http://portal.acm.org/citation.cfm?id=1858171.1858195&coll=portal&dl=ACM&type=series&idx=SERIES378&part=series&WantType=Proceedings&title=DIS&CFID=105514832&CFTOKEN=10835187 online].</ref> where participants produce vocal imitations to mimic the sonic behavior of objects while they are being interacted with;
* theatrical practices, such as theatrical metaphors and dramatic performance;<ref>Sandra Pauletto, Daniel Hug, Stephen Barrass and Mary Luckhurst (2009). ''Integrating Theatrical Strategies into Sonic Interaction Design''. In: Proceedings of Audio Mostly 2009 – 4th Conference on Interaction with Sound (2009), Glasgow, 6 p. Available: [http://sid.soundobject.org/export/775/action/doc/AudioMostlyPaulettoHugBarrassLuckhurstCameraReady.pdf PDF] {{webarchive|url=https://web.archive.org/web/20110827064759/http://sid.soundobject.org/export/775/action/doc/AudioMostlyPaulettoHugBarrassLuckhurstCameraReady.pdf |date=2011-08-27 }} and [http://www.citeulike.org/group/10826/article/7009520 online]</ref>
* basic design,<ref>Davide Rocchesso, Pietro Polotti, and Stefano delle Monache, (28 December 2009). "Designing Continuous Sonic Interaction". ''International Journal of Design'' 3(3). Available: [http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/620 online] and [http://rice.iuav.it/138/ PDF]</ref> based on demonstrations and [[Philosophy:Intersubjectivity|intersubjectivity]];
* video prototyping<ref>Wendy E. Mackay and Anne Laure Fayard, (1999). ''Video brainstorming and prototyping: techniques for participatory design'', Pittsburgh, Pennsylvania: Conference on Human Factors in Computing Systems, CHI '99 extended abstracts on Human factors in computing systems, {{ISBN|1-58113-158-5}}. Available: [http://portal.acm.org/citation.cfm?id=632716.632790&coll=GUIDE&dl=GUIDE&type=series&idx=SERIES260&part=series&WantType=Proceedings&title=CHI&CFID=103416465&CFTOKEN=62376791 online]</ref> with sonic overlays;
* foley artistry in [[Engineering:Filmmaking|filmmaking]];<ref>Michel Chion, (1994). ''Audio-Vision: sound on screen''. New York: Columbia University Press, {{ISBN|0-231-07898-6}}, {{ISBN|0-231-07899-4}}. [http://filmsound.org/philips.htm Book review on f ilmsound.org]</ref>
* acting out sound dramas.<ref>Daniel Hug, (2010). "Investigating Narrative and Performative Sound Design Strategies for Interactive Commodities". ''Lecture Notes in Computer Science'', Volume 5954/2010:&nbsp;12-40, {{doi|10.1007/978-3-642-12439-6_2}}. Available: [https://springerlink3.metapress.com/content/f7xv47j4u4123x72/resource-secured/?target=fulltext.pdf&sid=zhfkgzmf04koyx45k3zznu55&sh=www.springerlink.com online] {{webarchive|url=https://archive.today/20121217185143/https://springerlink3.metapress.com/content/f7xv47j4u4123x72/resource-secured/?target=fulltext.pdf&sid=zhfkgzmf04koyx45k3zznu55&sh=www.springerlink.com |date=2012-12-17 }}.</ref>

=== Interactive art and music ===

In the context of sonic interaction design, interactive art and music projects are designing and researching aesthetic experiences where sonic interaction is in the focus. The creative and expressive aspects – the aesthetics – are more important than conveying information through sound. Practices include installations, performances, public art and interactions between humans through digitally-augmented objects/environments. These often integrate elements such as embedded technology, gesture-sensitive devices, speakers or [[Context awareness|context-aware]] systems.

The experience is in the focus, addressing how humans are affected by the sound, and vice versa. Interactive art and music allows researchers to question existing paradigms and models of how humans interact with technology and sound, going beyond paradigms of control (human controlling a machine). Users are part of a loop which includes action and perception.

Interactive art and music projects invite explorative actions and playful engagement. There is also a multi-sensory aspect, especially haptic-audio<ref>The fifth International Workshop on Haptic and Audio Interaction Design (HAID) on September 16–17, 2010 in Copenhagen, Denmark. http://media.aau.dk/haid10/?page_id=46</ref> and audio-visual projects are popular. Amongst many other influences, this field is informed by the development of the roles of instrument-maker, composer and performer merging.<ref>International Conference on New Interfaces for Musical Expression http://www.nime.org/</ref>

Artistic research in sonic interaction design is about productions in the [[Interactive art|interactive art]]s and performing arts, exploiting the role of enactive engagement with sound–augmented interactive objects.<ref>John Thompson, JoAnn Kuchera-Morin, Marcos Novak, Dan Overholt, Lance Putnam, Graham Wakefield, and Wesley Smith, (2009). "The Allobrain: An interactive, stereographic, 3D audio, immersive virtual world". ''International Journal of Human-Computer Studies'' 67(11) (Nov. 2009):&nbsp;934–946. {{doi|10.1016/j.ijhcs.2009.05.005}}</ref>

=== Sonification ===

[[Physics:Sonification|Sonification]] is the data-dependent generation of sound, if the transformation is systematic, objective and reproducible, so that it can be used as scientific method.<ref>Sonification – A Definition http://sonification.de/son/definition</ref>

For sonic interaction design, sonification provides a set of methods to create interaction sounds that encode relevant data, so that the user can perceive or interpret the  conveyed information.   Sonification does not necessarily need to represent huge amounts of data in sound, but may only convey one or few data values in a sound. To give an example, imagine a light switch that, on activation would create a short sound that depends on the electric power consumed through the cable: more energy-wasting lamps would perhaps systematically result in more annoying switch sounds. This example shows that sonification aims to provide some information by using its systematic transformation into sound.

The integration of data-driven elements in interaction sound may serve different purposes: 
*  to allow the users to refine their actions via auditory feedback. Example: the sonification-enhanced drilling machine<ref>Tobias Grosshauser and Thomas Hermann, (2010). "Multimodal closed-loop Human Machine Interaction" Proc. Interactive Sonification Workshop, Stockholm, http://interactive-sonification.org/ISon2010/proceedings</ref> which indicates by sound when a wanted orientation to the wall is reached.
* to influence how humans perceive their own body movement.<ref>{{cite journal |last1=Effenberg |first1=Alfred O. |title=Movement sonification: Effects on perception and action |journal=IEEE MultiMedia |date=2005 |volume=12 |issue=2 |pages=53–59 |doi=10.1109/MMUL.2005.31 |s2cid=5211383 }}</ref>
* to create a sonic gestalt for the interaction which allows users to compare the detailed performance on repeated interactions: for instance rowing strokes may be sonified so that the sportsmen can better synchronize their action.
* to enable novel functions that would otherwise be not available (e.g. a bottle that displays by sound how much fluid is poured into glasses so that the users can more easily fill the equal amount of liquid in different glasses).

Within the field of sonification, sonic interaction design acknowledges the importance of human [[Human–computer interaction|interaction]] for understanding and using auditory [[Feedback|feedback]].<ref>Thomas Hermann, and Andy Hunt, (2005). "Guest Editors' Introduction: An Introduction to Interactive Sonification". ''IEEE MultiMedia'' 12(2):&nbsp;20-24. {{doi|10.1109/MMUL.2005.26}}</ref> Within sonic interaction design, sonification can help and offer solutions, methods, and techniques to inspire and guide the design of  products or interactive systems.

== See also ==
{| width="100%"
|- valign=top
|width="33%"|
* Ecomusicology
* List of human-computer interaction topics
* [[Design]]
* [[Engineering:Haptic technology|Haptic technology]]
* Social interaction
* [[Interactivity]]
|width="33%"|
* [[Interaction design]]
* Interactive systems engineering
* [[Interaction design pattern]]
* [[3D interaction|3D interaction]]
* [[Human–computer interaction]]
* [[Sound and music computing]]
|width="33%"|
* [[Physics:Soundscape|Soundscape]]
* [[User interface]]
* [[User interface design]]
* [[User-centered design]]
* [[Usage-centered design]]
* [[Virtual reality]]
|}

== References ==
{{Reflist}}

== Further reading ==
* Franinović, K., and Serafin, S., Eds. ''Sonic interaction design''. The MIT Press, Cambridge, Massachusetts, 2013.
* Stefano Delle Monache, Pietro Polotti, Davide Rocchesso (2010). ''A Toolkit for Explorations in Sonic Interaction Design''. In: Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound, 2010, New York (AM '10), {{ISBN|978-1-4503-0046-9}}, {{doi|10.1145/1859799.1859800}}, [http://portal.acm.org/citation.cfm?id=1859800 citation]
* Eoin Brazil and Mikael Fernström, (2009). ''Empirically Based Auditory Display Design''. In: Proceedings of the SMC 2009 - 6th Sound and Music Computing Conference, 23–25 July 2009, Porto, Portugal. Available: [http://smc2009.smcnetwork.org/programme/pdfs/149.pdf online].
* Karmen Franinović, Yon Visell, Daniel Hug, (2007). ''Sound Embodied: A Report on Sonic Interaction Design in Everyday Artifacts''. In: Proceedings of the 13th International Conference on Auditory Display, Montréal, Canada, June 26–29, 2007. Available: [http://www.icad.org/Proceedings/2007/FraninovicHug2007.pdf online].
* Ernest A. Edmonds, Alastair Weakley, Linda Candy, Mark Fell, Roger Knott, and Sandra Pauletto, (2005). "The Studio as Laboratory: Combining Creative Practice and Digital Technology Research". ''International Journal of Man-Machine Studies'' 63(4–5):&nbsp;452–481. Available: [http://www.informatik.uni-trier.de/~ley/db/journals/ijmms/ijmms63.html#EdmondsWCFKP05 online].
* Ernest Edmonds, Andrew Martin, and Sandra Pauletto, (2004). ''Audio-visual Interfaces in Digital Art''. In: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology (ACE '04), Singapore, June 3–5, 2004, pp.&nbsp;331–336, {{doi|10.1145/1067343.1067392}}. Available: [http://portal.acm.org/citation.cfm?doid=1067343.1067392 online].
* Thomas Hermann and Andy Hunt, (2004). ''The Importance of Interaction in Sonification''. In: Proceedings of ICAD Tenth Meeting of the International Conference on Auditory Display, Sydney, Australia, July 6–9, 2004. Available: [http://www.icad.org/websiteV2.0/Conferences/ICAD2004/papers/hunt_hermann.pdf online].
* Niklas Röber and Maic Masuch, (2004). ''Interacting With Sound: An Interaction Paradigm for Virtual Auditory Worlds''. In: Proceedings of ICAD Tenth Meeting of the International Conference on Auditory Display, Sydney, Australia, July 6–9, 2004. Available: [http://www.icad.org/websiteV2.0/Conferences/ICAD2004/papers/roeber_masuch.pdf online].

== External links ==
{| width="100%"
|- valign=top
|width="50%"|
*[http://smcnetwork.org/ Sound and Music Computing]
*[http://www.interactive-sonification.org/ Interactive Sonification]
*[http://isam.eecs.qmul.ac.uk Interactional Sound and Music]
|width="50%"|
*[http://sid.soundobject.org COST Action on Sonic Interaction Design]
*[https://web.archive.org/web/20100117224728/http://sonic-interaction.com/ Sonic Interaction Portal]
|}

{{Design}}

{{DEFAULTSORT:Sonic Interaction Design}}
[[Category:Multimodal interaction]]
[[Category:Sound]]

{{Sourceattribution|Sonic interaction design}}