Bird Sounds

RESEARCH STATEMENT

Bird Sounds (Williams 2022) is a short film that examines grief through sound and memory and is particularly concerned with exploring novel uses of sound in film narrative. It was created as part of an interdisciplinary research project investigating sound-evoked visual memory in fiction film. Joel, the protagonist, is an acoustic ecologist who records soundscapes, and documents his family life through sound recordings. Unable to process the death of his son, Joel returns to the jungle seeking out echoes of sounds they recorded together, while at home he listens to recordings he made documenting his family life. This process slowly unlocks Joel’s ability to accept and engage with his loss as the sounds trigger vivid memories. The film was inspired in part through listening to audio recordings that I make of my daughter and son as they are growing up, and their capacity to evoke strong visual memories. The subject of grief is drawn from the difficult experience of a friend who lost her husband, and a broader meditation on how we might process loss through returning, almost ritualistically, to places of meaning and memory. Underlying all of this is an exploration of sound in the construction of film narrative and image memory.

The film explores the complex interplay between auditory perception and subjective experience, particularly through the use of recordings and headphones as a narrative device. The protagonist's frequent use of headphones to listen to recordings introduces a layer of ambiguity for the audience. Are we listening to a recording, reliving a past moment through the protagonist's sound memories, or perhaps experiencing a new recording being made? The question of what Michel Chion describes as internal sound (2019) is also raised and, coupled with the sparse use of dialog, foregrounds the malleability of sound within film narrative, where sound might be at once subjective and objective.

The opening of the film demonstrates this; we see our protagonist Joel walking through the jungle, accompanied by the rich sound of jungle life. This sound is then muted when he puts on his headphones, suggesting we were hearing through Joel’s ears from the beginning. The sound world erupts again when he begins his recording; the 5.1 surround sound field shifting as he moves his directional microphone around the scene. The audience hears the sound move around the theatre as the microphone moves, suggesting a subjective rendering of the scene but leaving this interpretation open ended.

This ambiguity of sound is designed, in part, as a metaphor for the fluid nature of memory and how sound might suggest image memories. That leads to questions on how sound might affect an audience’s memories of the images they see.

Apart from writing and directing this film, my typical role within film productions is as a sound designer and composer. Because of this I am naturally concerned with the power of sound and music to shape our experience of film narratives. Strongly related to this is the potential for sound and music to alter our visual perception of the image track. Do we sometimes “see” things that we have only heard when watching a film? This thought led to discussions with cognitive psychologists to develop an interdisciplinary research project asking the question: “Do we remember seeing things that were presented by sound alone when watching a film?” Bird Sounds is the creative practice output of the research project and was created for use in the cognitive study.

It is common cinematic practice to have events represented by sound alone when events are typically occurring off screen or otherwise occluded from view. Audiences, who are active participants in the construction of the narrative they experience, are regularly tasked with “filling in material” and “adjusting what they remember” (Bordwell 1985, 35) in the process of viewership. Anecdotal evidence suggests that some audiences recall viewing things that they had only heard when watching film (Hanich 2010; Louden 2020), and there is a growing body of scholarship concerning false visual memories, including those invoked by sound (Lindner and Henkel 2015; Loftus and Pickrell 1995). Sound has been shown to evoke images in controlled experimental conditions (Shams, Kamitani, and Shimojo 2000).

For our study, two versions of Bird Sounds were made, each with the same soundtrack, but differing on what was presented visually. Audiences viewed one version of the film and then were asked if they remembered seeing certain (test) events. For example, in the opening scene some audiences saw and heard Joel hitting the record button on the audio recorder and others didn’t see this but still heard the event. In a post-screening questionnaire, they indicated their recall confidence of seeing the event via a sliding scale. This is the first study of its kind to try to discover if an image memory might be evoked by sound when experiencing a movie. Our findings suggest that sound can evoke visual memories of unseen events for some audience members in some cases (Williams, Maus, and Styles 2023). It is not clear if this “false” visual memory is created at the time of viewing or created at the time of recall.

Bird Sounds represents the creative practice output of a novel interdisciplinary research project that has been screened at festivals internationally. It is a film concerned with grief, memory and sound, created and funded for use in a cognitive research project investigating sound and visual memory. It represents a different avenue of screen practice research.

PEER REVIEW 1

There is much to commend in this work, both the creative output and the exegesis.

The interdisciplinary collaboration between the researchers is impressive and enlightening in their exploration of how sound evoked memory studies informs screen fiction narrative. The work will be of interest to filmmakers, sound designers and production practitioners, interdisciplinary creative media, and memory studies researchers. The work will also be an excellent teaching and learning resource in the screen production training and education sectors in its study of the construction of narrative and point of view through sound and image memory. The research has implications for how screenwriters/filmmakers might approach writing sound as narrative.

The statement is well written and presents a cohesive argument supported by case study and audience qualitative research studies. The research question is clearly articulated, and the author succinctly maps the creative works development process within a theoretical framework that informs both submission components. The film is well produced. The mise-en-scène style and form elements – cinematography, sound design/production, production design, editing and performances of the actors – are excellent. The author delivers what he has set out to do. The film’s representation of grief and loss through the manipulation of sound to indicate shifting point of view and the internal state of the protagonist is engaging and suspenseful as we journey with him and his hauntings through “the complex interplay between auditory perception and subjective experience.”

As creative practice led research the questions for investigation are clearly framed within theoretical frameworks, drawing on formalist film theory, memory, and sound studies. The findings are integrated into the analysis and used to interrogate how these practices are applied to the creative work. The author includes reflections on his own practice, incorporates personal experiences and technical expertise to consider the research outputs in relation to existing knowledge and creation of new knowledge through the production of the creative artefact and exegesis (Skains 2018).

I would like a stronger [research] statement in the conclusion regarding the significance of interdisciplinary collaborations and the need for more research (as demonstrated through this study) into how sound contributes to screen narrative, affect and audience experiences. And how other disciplines, such as memory studies might contribute and inform our understanding of the complexities of sound and visual memories on screen.

This collaboration offers a potential model for further work to be done in this area. 

PEER REVIEW 2

The submission presents a creative work in the form of a film where the notion of “watching a film” is challenged. In Bird Sounds (Williams 2022), although you do “watch” the film, it is the soundtrack and sound design that takes the lead of the narrative. Being unique in this way, the audience is challenged to consider what is seen, what is heard and what memories are evoked – and when. As noted by the author, “the film explores the complex interplay between auditory perception and subjective experience.” The interdisciplinary nature of the film is to be commended as it successfully combines a quality creative work with cognitive psychological underpinnings.

The submission is successful in achieving the desired outcomes. In addition to being an interdisciplinary research project asking the question: “Do we remember seeing things that were presented by sound alone when watching a film?” The film stands as an individual artefact that has already been screened at international festivals. The accompanying research statement provides valuable insights into how the project came about, and the intent, and this is evident in the creative work.

The research provides evidence of questions on how sound might affect an audience’s memories of the images they see. This is successfully realised through the explicit use of surround/ immersive sound and key edits to the soundtrack – either through timing, layering, or mixing. The use of the soundtrack to take lead on the narrative for the entirety of the film is innovative, and situates the audience sonically in the first-person, often at times in juxtaposition to the imagery. This is a contextualised example in relation to Michel Chion’s description of “internal sound” (2019, 76). The author also acknowledges other contextual sources aligned to the field of research.

The research presents a unique film experience for the audience where the soundtrack provides most of the narrative information, despite limited dialogue. This was a pleasure to experience.

My only suggestion [for revising the research statement or creative work] would be to have a disclaimer stating to experience the film as intended, audiences need to experience the film in either 5.1 audio format or binaural through headphones. 

RESPONSE TO PEER REVIEW

I am extremely gratified with the response from the two peer reviewers, and I appreciate their time and thoughtful comments. Reviewer one’s comment that

the work will be of interest to filmmakers, sound designers and production practitioners, interdisciplinary creative media, and memory studies researchers. The work will also be an excellent teaching and learning resource in the screen production training and education sectors in its study of the construction of narrative and point of view through sound and image memory

was particularly gratifying as this was one of my hoped-for goals for this work. This comment further encouraged me to develop a pedagogical resource that can be used by educators to discuss the interdisciplinary themes of this research and the roles of sound in film narrative. This will be linked to my personal website www.fluidsound.com for those interested. The film versions and other resources that were used for the study are available for other researchers to use and are available in the digital repository linked to this research.

I have made some minor changes to the research statement following the suggestions of the reviewers. Specifically, I included a final paragraph highlighting the significance of this kind of interdisciplinary collaboration and the need for more of them. I have also added a suggestion that the viewer use headphones when watching the film so as to experience the binaural immersive soundtrack to its fullest.

REVISED RESEARCH STATEMENT

The online version of this film is rendered binaurally to provide an immersive sound experience. It is recommended to view using headphones.

Bird Sounds (Williams 2002) is a short film that examines grief through sound and memory and is particularly concerned with exploring novel uses of sound in film narrative. It was created as part of an interdisciplinary research project investigating sound-evoked visual memory in fiction film. Joel, the protagonist, is an acoustic ecologist who records soundscapes, and documents his family life through sound recordings. Unable to process the death of his son, Joel returns to the jungle seeking out echoes of sounds they recorded together, while at home he listens to recordings he made documenting his family life. This process slowly unlocks Joel’s ability to accept and engage with his loss as the sounds trigger vivid memories. The film was inspired in part through listening to audio recordings that I make of my daughter and son as they are growing up, and their capacity to evoke strong visual memories. The subject of grief is drawn from the difficult experience of a friend who lost her husband, and a broader meditation on how we might process loss through returning, almost ritualistically, to places of meaning and memory. Underlying all of this is an exploration of sound in the construction of film narrative and image memory.

The film explores the complex interplay between auditory perception and subjective experience, particularly through the use of recordings and headphones as a narrative device. The protagonist's frequent use of headphones to listen to recordings introduces a layer of ambiguity for the audience. Are we listening to a recording, reliving a past moment through the protagonist's sound memories, or perhaps experiencing a new recording being made? The question of what Michel Chion describes as “internal sound” (2019, 76) is also raised and, coupled with the sparse use of dialog, foregrounds the malleability of sound within film narrative, where sound might be at once subjective and objective.

The opening of the film demonstrates this; we see our protagonist Joel walking through the jungle, accompanied by the rich sound of jungle life. This sound is then muted when he puts on his headphones, suggesting we were hearing through Joel’s ears from the beginning. The sound world erupts again when he begins his recording; the 5.1 surround sound field shifting as he moves his directional microphone around the scene. The audience hears the sound move around the theatre as the microphone moves, suggesting a subjective rendering of the scene but leaving this interpretation open ended.

This ambiguity of sound is designed, in part, as a metaphor for the fluid nature of memory and how sound might suggest image memories. That leads to questions on how sound might affect an audience’s memories of the images they see.

Apart from writing and directing this film, my typical role within film productions is as a sound designer and composer. Because of this I am naturally concerned with the power of sound and music to shape our experience of film narratives. Strongly related to this is the potential for sound and music to alter our visual perception of the image track. Do we sometimes “see” things that we have only heard when watching a film? This thought led to discussions with cognitive psychologists to develop an interdisciplinary research project asking the question: “Do we remember seeing things that were presented by sound alone when watching a film?” Bird Sounds is the creative practice output of the research project and was created for use in the cognitive study.

It is common cinematic practice to have events represented by sound alone when events are typically occurring off screen or otherwise occluded from view. Audiences, who actively participate in the creation of the narrative they experience, are regularly tasked with “filling in material” and “adjusting what they remember” in the process of viewership (Bordwell 1985, 35). Anecdotal evidence suggests that some audiences recall viewing things that they had only heard when watching film (Hanich 2010; Louden 2020), and there is a growing body of scholarship concerning false visual memories, including those invoked by sound (Lindner and Henkel 2015; Loftus and Pickrell 1995). Sound has been shown to evoke images in controlled experimental conditions (Shams, Kamitani, and Shimojo 2000) and film is often used as a metaphor to understand processes and concepts concerned with the formation and malleability of memory (Seamon 2015).

For our study two versions of Bird Sounds were made, each with the same soundtrack, but differing on what was presented visually. Audiences viewed one version of the film and then were asked if they remembered seeing certain (test) events. For example, in the opening scene some audiences saw and heard Joel hitting the record button on the audio recorder and others didn’t see this but still heard the event. In a post-screening questionnaire, they indicated their recall confidence of seeing the event via a sliding scale. This is the first study of its kind to try to discover if an image memory might be evoked by sound when experiencing a movie. Our findings suggest that sound can evoke visual memories of unseen events for some audience members in some cases (Williams, Maus, and Styles 2023). It is not clear if this “false” visual memory is created at the time of viewing or created at the time of recall.

Bird Sounds represents the creative practice output of a novel, artist lead, interdisciplinary research project that has been screened at festivals internationally. It is a film concerned with grief, memory and sound, created and funded for use in a cognitive research project investigating sound and visual memory.

The significant role of sound in shaping an audience's experience, understanding, and memory of screen narratives has been relatively underexplored in cognitive and film studies. This research is a step towards bridging the existing gap and highlights the importance of collaboration between practitioners and theorists to examine how audiences engage with screen media. International conference presentations of this research have led to valuable discussions and feedback. Importantly they affirmed the significance and need for more interdisciplinary approaches such as this one, and an enthusiasm for theorists to engage in them.

REFERENCES

Bordwell, David. 1985. Narration in the Fiction Film. Madison, WI: University of Wisconsin Press.

Chion, Michel. 2019. Audio-Vision: Sound on Screen. 2nd ed. Columbia University Press.

Hanich, Julien. 2010. Cinematic Emotion in Horror Films and Thrillers: The Aesthetic Paradox of Pleasurable Fear. New York: Routledge.

Lindner, Isabel, and Linda A. Henkel. 2015. “Confusing What You Heard with What You Did: False Action-Memories from Auditory Cues.” Psychonomic Bulletin & Review 22, no.6: 1791–97. https://doi.org/10.3758/s13423-015-0837-0.

Loftus, Elizabeth F, and Jacqueline E Pickrell. 1995. “The Formation of False Memories.” Psychiatric Annals 25, no. 12: 720–25. https://doi.org/10.3928/0048-5713-19951201-07.

Louden, Sarah. 2020. “Sound-Induced Visual Illusion in FIlm.” In Music in Action Film: Sounds Like Action! 1st ed., edited by James Buhler and Mark Durrand, 238-255. New York: Routledge.

Seamon, John G. 2015. Memory and Movies: What Films Can Teach Us about Memory. Cambridge, MA: The MIT Press.

Shams, Ladan, Yukiyasu Kamitani, and Shinsuke Shimojo. 2000. “What You See Is What You Hear.” Nature 408, no. 6814: 788–788. https://doi.org/10.1038/35048669.

Williams, Ross Adrian, Gerrit W Maus, and Suzy J Styles. 2023. “Sound Evoked Visual Memory in Narrative Film.” Preprint. PsyArXiv. https://doi.org/10.31234/osf.io/mhvq5.