Reflective readings that have
supported my research


Psychoacoustic Cues to Emotion in Speech Prosody and Music

Coutinho, Eduardo, and Nicola Dibben. 2013. ‘Psychoacoustic Cues to Emotion in Speech Prosody and Music’. Cognition & Emotion 27 (4): 658–84. https://doi.org/10.1080/02699931.2012.732559.
Psychoacoustic cues to emotion in speech prosody and music

by Eduardo Coutinho and Nicola Dibben

I found this article interesting, especially in terms of how to use sound to elicit specific emotions in the listener, in which the writer refers to as “emotions are communicated auditory domain.” The writer discusses how changes in auditory communication are similar to facial micro-expressions, and perceptions of these nesciences as readily understood by the listener.

The article speaks of hose some voice inflexion is readily understood across cultures and languages, and that music’s ability to convey emotions is universal.

The article contained a lot of technical information that at this point in time went over my head, but I can ascertain that with the perception of sound, fast music and/or speech can convey higher energy, but is perceived as tenser and as a result less pleasurable to listen too.

The study looked at the nuances of the sound, and the way the listener perceived it and spoke about how the musical structure was a neurological response. Music can elicit an emotional response, even if there are no spoken words. This study used a self-reporting method which in which the listener would record the effect of the sound they were exposed too. They focused on five main features of music, which included tempo, speech rate, pitch, melody/prosody contour and pitch variations, timbre, and roughness. They pointed out that people are known to listen to music to help mood regulation and pleasure over speech alone. The researchers speak about how this research can have other applications, such as for the use of diagnosing different psychiatric conditions, to improve hearing aid, and also human-computer interactions.

Reflection on practice 

The article has been useful in terms of how I deliver my information to the listener of my audio podcast. This means I have considered how to articulate my narration, how close I want to sit to the microphone, and what audio plugins I want to use to create the right ambience to tell my story.


Sonic Art An Introduction to Electroacoustic composition

Moore, Adrian. 2016. Sonic Art : An Introduction to Electroacoustic Music Composition. Routledge.

Sonic Art An Introduction to Electroacoustic composition

by Adrian Moore

Chapter One, What is sound. 

I have found this book very informative and it contains lots of information on the creation of composition inside of a computer. Right at the start, with the introduction ‘jumping in at the deep end’ the writer reminds us that we are only limited by our imaginations which is a point I made often to my children and university students and a mantra I implement for myself. The writer advises us that the shape of sound is like shaping clay, sometimes it’s formless, and sometimes it takes the form of a vase, and to also think of sound as musical over the music. That we should define sound with words, as this encourages analysis.

The writer of this book speaks of sonic art theory and talks about words you can use to describe sounds such as gesture, texture, utterance, energy, motions, space, and environment. The writer urges us to look at the computer as our instrument, and that practice is as essential as practising scales on the piano. Practice makes progress, after all.

We are also advised to consider the listener, we are primarily creating a sound that seems right to our ear, but it’s also essential to keep in mind who will be doing the listening, and what the end goal is. I generally follow my gut feeling of what seems right, but it is also essential to learn WHY it feels right.

The writer speaks of the importance of proper labelling of any music we make, following a practice that I have already started doing, using onomatopoeia alongside other qualities of the sounds feeling and perception.  These labels can come from the analytical investigation into what the quality of the sound sounds like.  These qualities can be reduced to emotions or real-world sounds. The book says to examine sound from all angles, and to ensure you record your audio to keep is pure as possible, and to aim for good quality stereo when capturing sound outside of the studio.

This writer of this book says first start with an idea, as this is where a composition often beings. I worked to shape and idea I have to create what my hearing experience is like. his will reply on the LR leaning sounds so will need headphones for a full experience. Where in which the listener can hear the pulsatile whoosh of my left ear, and high pitched whine of the tinnitus in the right, right along with the weird way my brain scrambles words once sound enters my, brain for processing. To keep the listener engaged, while also educating them to how another person perceives the world.

I particularly liked how the writer said ‘we are always reacting against what we perceive as any particular situation‘. I like the way this is phrased; sound can physically affect our psychology. The writer also speaks of the importance to keep adequate notes on the creation of our compositions, so that we can recreate this in the future.

Reflection on practice 

I resonated with the sentiment of starting with an idea, and have funnelled this into my own practice. I had the idea of creating a sound piece to convey how I hear the world, and built upon it.


Immersive Sound : The Art and Science of Binaural and Multi-Channel Audio

Immersive Sound: The art and science of binaural and multi-channel audio

Roginska, A., & Geluso, P. (Eds.). (2017). Immersive sound: the art and science of binaural and multi-channel audio.

Introduction, Agnieszka Roginska and Paul Geluso

Immersive Sound

I found this introduction very interesting. The section on immersive sound is of interest to me, especially what the writers mentioned surrounding how sound can be used in film to ground the audience in a location while the visuals in the movie can shift dramatically. The writers explain how a sense of space and be conveyed via the way the sound is recorded, such as recording gin a small space to convey a sense of intimacy. The writers speak about how you can create a soundscape that will wrap the listener by using a mix of directional and non-directional sound, such as reverb on sounds to convey big space.

Soundscapes are layered; for instance, when I was reading this article, I was getting my hair coloured at the salon. I could hear the traffic from the road, the airconditioning, people in the salon talking to each other, the music playing, the sound of my pen tapping on my iPad, and Lou (My sister in law) moving stuff around in the back of the building, and the sound of the builders from across the road. I took a voice memo of the soundscape. From my understanding, I could recreate this soundscape inside my computer by placing together the separate layers and applying different effects to emulate the distance from the listener, and what surfaces the sounds are bouncing off.

The Listening Experience

A recreated audio experience, all the sounds recorded and stored separately, then reproduction is created of the natural listening environment. Natural listening is creating listening environments that mimic the sound of the real world, in a way that could only be experienced in a virtual space.

Listener’s Perspective

The heard information used to assess the space and distance between the sound and the listener. Fixed perspective is monophonic, or stereophonic, surround sound, like in our sound is multiple speakers surrounding the listener to recreate a complete listening environment. Playback and sound reproduction have become more sophisticated with a move towards creating an audio experience that does not rely on the listener’s prior knowledge to make sense of what they are hearing.

About this book

This book discusses creating immersive soundscapes and how different production and reproduction can result in varying the listening experience.

Reflection on practice 

I implemented a lot of what I learned in this paper into my work, especially the section that recreates the ambience of listening to a friend talk in a busy cafe.


Designing Sound for Animation

Chapter 1-3
Beauchamp, Robin. 2013. Designing Sound for Animation. Oxford, UNITED KINGDOM: CRC Press. http://ebookcentral.proquest.com/lib/aut/detail.action?docID=1157737.
The first chapters of this book have been great to help reinforce sound knowledge I require — the first covering the perception of sound. Sound in animation is used to guide the audience into listening to what is essential for the stories narrative. It speaks bout Localization and how it can be used to place objects in reality. This is an effect I wish to employ for my sound project, as I will be recreating what it is like for me with two different types of tinnitus.
The book covered effects such as reverb and delay in simple terms helping me understand how I can use these with my work.
Such as having a piece of audio recorded in a soundproof room but supposed to be sounding like the speaker in a large cavern. Different types of resets can help recreate this sound and help the viewer stay immersed in the story. Years ago, I was part of a group that attempted to dub some audio in a film project, and it ended up being terrible. Hindsight I realise that we should have taken the recorded sound and used effects so that the speakers matched the environment that they were in.
They spoke about how light travels faster than sound, so when syncing audio to animation care must be taken to ensure that the audio is offset slightly, so that the viewer sees the action before they hear it, like with Thunder and Lightening.  I read about sampling rates, and how to ensure that you capture the best quality sound, including why we need things like a pop filter, and other reasons why you might have trouble recording the voice-overs for animations and when you can, and when you can’t compress audio.
Sound Casslifactions

I learned about Michel Chions Chion Classifications, with sound is classified under Causal, semantic, and reduced sounds. Casual meaning you hear anything you see on TV, but you don’t necessarily see everything you hear, semantic is any sound that conveys language, and reduced sounds are sounds that have been stripped down and paired with something entirely different.  I also read about Diegetic (sound perceived by characters in the film) and non-diegetic sound (sound only perceived by the audience ie canned laughter).

Sound can ben used to Establish or clarify a point of view, with an approach known as Leitmotif, i.e. music that has been assigned to a specific character, this is an effect that has been borrowed from Opera, and from memory is used in Fantasia. Sound can also be used to underscore the subject of a storyline and to contrast reality from fantasy scenes. Meaning you could have a montage sequence which removes all diegetic sound, which in turn can remove the characters from the reality of the film. Off-screen sounds can be used to indicate events that the watcher can not see, used in comedy to show that a character has fallen downstairs, You might hear the sounds of crashing and banging, or witnessing another character react to this sound.

Sound can is also useful for continuity of the film, bridging together different scenes, and by linking emotional continuity. Sound can also be used to position a character within the framework of the film.

Dialogue

The dialogue for animation is recorded before the characters are animated. There is a technique for directing voice talent, as acting for animation can be vastly different from acting for live-action. The director has to be prepared for many different scenarios.

Once the dialogue is captured, it is evaluated and edited so that distortions such as sibilance the Sssss sounds, are reduced. The book mentions that experienced voice actions know to put the inflexion on a different part of the work. The example in the text refers to the different way of saying Snake. “Sssssssnake” to “Snaaaaaake”.

Reflection on practice 

This book has been a fantastic resource for continuing my education on the use of sound in animation. There are many facets that I had not thought about and has opened up many avenues of research, such as using sound to create continuity, reinforce the narrative, and convey emotion.  Sound is more than communicating spoken language information to the audience; sound creates the 4th dimension of viewership.


Test-Retest Reproducibility of Response Duration in Tinnitus Patients With Positive Residual Inhibition

Deklerck, Ann N., Sofie Degeest, Ingeborg J. M. Dhooge, and Hannah Keppler. 2019. ‘Test-Retest Reproducibility of Response Duration in Tinnitus Patients With Positive Residual Inhibition’. Journal of Speech, Language & Hearing Research 62 (September): 3531–44. https://doi.org/10.1044/2019_JSLHR-H-18-0514.
This short paper discusses how noise frequencies can help mask the sound of tinnitus. They tested patients with subjective tinnitus and attempted to find a frequency that would mask the symptoms. This paper interested me because there is so little research into the different causes of tinnitus and for many sufferers, the only advice Doctors can give them is to undertake counselling and tinnitus retraining therapy. This study concluded that using specific frequencies of white noise has excellent reliability to inhibit the adverse side effects of suffering from tinnitus.

Reflection on practice 

As a person who suffers from two types of tinnitus, it was interesting to learn more about my condition. I  have previously stayed away from googling this topic as I have been told that it doesn’t help those who have tinnitus and can sometimes result in the sufferer from enduring more anxiety about it. It turned out that for me, understanding why I have tinnitus has helped me come to terms with why I hear it all the time. I also learned that recreating the sound frequency of my tinnitus meant I couldn’t listen to it through the speakers in my right ear and subtly reduced my tinnitus symptoms.

A Review of Auditory Prediction and Its Potential Role in Tinnitus Perception

Durai, Mithila, Mary G. O’Keeffe, and Grant D. Searchfield. 2018. ‘A Review of Auditory Prediction and Its Potential Role in Tinnitus Perception’. Journal of the American Academy of Audiology 29 (6): 533–47. https://doi.org/10.3766/jaaa.17025.
This paper talks about a recent proposition surrounding auditory prediction errors and related memory representations, which may play a role in what causes tinnitus.
It is from this paper that I took the line Tinnitus can be described as the perception of sound in the absence of noise. In the first line of the introduction, they explain it almost the same as this, referring to the perception of sound in the absence of sound in the environment. This is what it is like inside my head, no matter what other sounds are going on, I always hear ringing in my right ear and a whoosh in my left.
I found it interesting that they speculate that tinnitus distress may be dependent on higher-order decision-making processes, as I have several processing disorders, one of which being auditory.
I learned the reason I hear this ringing is linked to something akin to muscle memory in the brain. My brain is used to receiving this frequency from this part of my ear, and since this part of my ear is now damage my brain is going haywire and creating the sound in my head all the time. It hears damage that the brain is receiving as broken signals and being interpreted as tinnitus.

Reflection on practice 

Not only did this paper give rise to the line referring to tinnitus is described as but also inspired what I have named my piece: The sound of NO silence.

Hearing Screen Animation

‘Hearing Screen Animation’. 2009. Metro Magazine: Media & Education Magazine, no. 161: 158.
This article speaks of how there is an increasing interest in writing papers about the sound design theory, but there are still (as of 10 years ago) little links drawn between animation and sound. This paper refers to sound as being the link between the story and space, allowing the narrative to leap out from the screen and into the viewer’s imagination and creating as I like to think of it, the 4th dimension.
The author, Rebecca Coyle, speak so of how fast turn around has effected sound in animation, viewing it with a dismissive attitude and treating is an afterthought, much like I have done in the past. This has had a roll-on effect of there being less space to experiment and as a result of less research into the subject.
Coyle says sound reaches out of the space of the animation, and that sound is difficult to contain, and unlike visual media, the sound is impossible to freeze, it is in constant motion.  Because of this sound adds depth a story that can not be ascertained by images alone.
Coyle makes an essential point that there is no native ambient sound in animation, every sound heard by the viewer MUST be created by the sound design, which means it is even more critical that adequate consideration is given to the sound aspect of the story. On the subject diegetic and non-diegetic sound, Coyle outlines that ALL action that has an associated sound must be heard, whereas not all sound we hear has to be seen.

Reflection on practice 

Treating sound as an essential aspect of animation, as crucial as the images we see will enrich my animation in a way that can not be sought from a picture alone. From this article, I have taken to heart the concept that there is no pre-existing ambient sound in animation, and ample consideration should be given to what sounds the audience is receiving.

Bibliography

Beauchamp, Robin. 2013. Designing Sound for Animation. Oxford, UNITED KINGDOM: CRC Press.
http://ebookcentral.proquest.com/lib/aut/detail.action?docID=1157737.
Coutinho, Eduardo, and Nicola Dibben. 2013. ‘Psychoacoustic Cues to Emotion in Speech Prosody and Music’. Cognition & Emotion 27 (4): 658–84. https://doi.org/10.1080/02699931.2012.732559.
Deklerck, Ann N., Sofie Degeest, Ingeborg J. M. Dhooge, and Hannah Keppler. 2019. ‘Test-Retest Reproducibility of Response Duration in Tinnitus Patients With Positive Residual Inhibition’. Journal of Speech, Language & Hearing Research 62 (September): 3531–44. https://doi.org/10.1044/2019_JSLHR-H-18-0514.
‘Design, When Everybody Designs : An Introduction to Design for Social Innovation’. n.d. Accessed 18 October 2019. http://eds.a.ebscohost.com.ezproxy.aut.ac.nz/eds/ebookviewer/ebook/bm[email protected]sdc-v-sessmgr02&vid=0&format=EK&rid=1.
Durai, Mithila, Mary G. O’Keeffe, and Grant D. Searchfield. 2018. ‘A Review of Auditory Prediction and Its Potential Role in Tinnitus Perception’. Journal of the American Academy of Audiology 29 (6): 533–47. https://doi.org/10.3766/jaaa.17025.
‘Hearing Screen Animation’. 2009. Metro Magazine: Media & Education Magazine, no. 161: 158.
‘Immersive Sound : The Art and Science of Binaural and Multi-Channel Audio: Live’. n.d. Accessed 12 October 2019. http://eds.a.ebscohost.com.ezproxy.aut.ac.nz/eds/detail/detail?vid=0&sid=1890dd6e-2203-45cd-9c60-99869e4419a8%40sessionmgr4008&bdata=JnNpdGU9ZWRzLWxpdmU%3d.
Moore, Adrian. 2016. Sonic Art : An Introduction to Electroacoustic Music Composition. Routledge.