Sounds+hands movements

Where -  Master’s student at University College London
When - 2013/2015
How - Prototyping, sonic interaction design, public speaking, quantitative user research.

Background

When interacting with physical objects, we receive sensory feedback about the characteristics of these objects. Current developments in interactive systems are opening up new avenues in the use and design of both physical and virtual objects. These developments have the potential to change people’s behaviours, perceptions and emotions – elements of user experience that can be measured. Soon, consumers might simply use gesture interaction to activate an invisible virtual volume control. 

Google's project Soli

Google's project Soli

In a nutshell

In my dissertation, I explored how the sounds resulting from tapping on a surface inform:

  1. The applied strength when tapping.
  2. The user’s ability to tap and changes in their emotional states.
  3. The physical features of the surface material (hardness).

The effect of sounds feedback has been understudied. The sounds resulting from touching a surface provides information about the material of the surface and about one's own touching behaviour, but it’s still unclear how this information impacts user’s emotions and behavior. Above all, there isn’t a shared framework to evaluate these potential effects.

The findings allowed me to validate hypotheses, provided basis for future research and for a framework to evaluate the effect of sounds feedback on user experience.

My role

  • Generated research ideas together with the principal investigator and reviewed relevant academic research
  • Selected research methods to properly investigate the problem and wrote the questionnaire material and test script
  • Built the prototype with Arduino and MAX-MSP
  • Set up the laboratory and conducted 20+ one-to-one sessions
  • Analysed data and written the final dissertation
  • Co-written the academic paper for publication
  • Presented the paper at UCD 2015 in October 2015

Process and examples

After generating hypotheses and selecting measures, a prototype of a tool to add immediate sound feedback to user’s tapping movement was built. MAX-MSP, Arduino, and movement and sound sensors were used to build the prototype.

A scheme showing the functioning of the prototype

A scheme showing the functioning of the prototype

 

The prototype was used to assess the impact of sound feedback on users’ emotions, behaviour and perception.  Tests were one-to-one sessions where about 20+ users were asked to tap on a real or an imagined 'virtual' surface while hearing different auditory feedback. 
 

The two 'surfaces' 

The two 'surfaces' 

Participants were presented with a questionnaire and their physiological responses (heart rate, skin conductance) were also recorded during the tests. Quantitative data was extracted with MatLab  a series of analyses were run with SPSS.

Measuring and analysing

Measuring and analysing

Awards/talks

 

The presentation I gave at UCD 2015


Read another one of my projects


Download my CV

It’s here

Contact me

Email: enrico.furfaro@gmail.com

Linkedin: here

Skype: enrico.furfaro