Entangled

RESEARCH STATEMENT

​

 “VR is where film was in 1915”

Scott Francis, Chief Technology Officer, THX – September 2016

​​

Research Background

​

As a screenwriter and film maker I was drawn to VR when content started to become more freely available at international film festivals in 2017-2018. However, I felt constantly disappointed at the experiences on offer, which didn’t seem to be making the best use of the technology and the immersive environment it could create. In particular, VR filmmakers were constantly using “tricks” to make sure the viewer was looking where the creator required them to look at any given moment. It occurred to me that many of these new experiences could be compared to the works of early filmmakers who were exploring the new technology of film with their existing knowledge of drama from the world of theatre: the “grammar “of filmmaking had yet to be developed.

The making of Entangled – with the help of a small innovation grant from the School of Creative Industries, University of Newcastle – had two main research objectives. Firstly, to explore how the new medium could be fully used in the service of dramatic storytelling; and secondly to investigate the best way to “script” the experience, given that, while the action of a VR drama remains linear, it takes place all around the viewer, a point of difference that is not easily accommodated with traditional screenplay conventions.

​

The Experience

​

Entangled was conceived as more than just a 360 film: it is designed to take place in an easily constructed “venue”, with each audience member asked to choose between an orange and a blue drink. Once he/she has done this, a headset is provided (the orange and blue experiences are different), and the immersive experience begins. The first thing the viewer sees in the virtual world is other people choosing their drink and putting on headsets. The main drama then begins.

​

Contribution & Significance

​

Entangled breaks new ground by providing an immersive experience that does not rely upon directing the gaze of the viewer. In a sense it is more like “real life” drama: if a bomb explodes in a shopping centre while you are there, you may be facing the explosion or not. Depending on where you are, and your orientation, your experience of a dramatic event is different but – with careful design – no less powerful. Entangled finds a way to provide immersive drama using the idea of the “special witness” – who is placed in the middle of things.

Furthermore, the development of Entangled trialled several different approaches to script development from a professional practitioner’s perspective, discovering important insights into the kinds of tools and processes required for the development of a VR experience with ambisonic audio. A key insight is that the traditional script is still required for actors, but is of limited use for the director, production designer and cinematographer, where six column scripts were developed to identify what was happening in each of north, south, east, west, zenith and nadir sections of the space.

​

Links

​

There are two versions of Entangled to witness. Before choosing, imagine you are in a dilapidated basement, looking for an experience of the sublime. You’ve paid a lot of money for this edgy – perhaps illegal – trip that uses the latest quantum technology. You are offered a choice of drink: orange or blue. Your decision…

​

https://youtu.be/0rHUZt_Cxas   Entangled (Blue)

​

https://youtu.be/vG0vWzbhGfU  Entangled (Orange)

​

​

PEER REVIEW 1

​

Entangled is an experiment in immersive storytelling. The author builds on his experience as a screenwriter to expand narrative design into the frameless format. The short film, ~9 min in duration, is produced as a live-action 360-degree stereoscopic film for Virtual Reality headsets. Within the experience, the viewer is free to direct his gaze in whatever direction they choose, while the narrative plays out in a linear fashion all around them. There are three aspects that are relevant and of interest: a) the experience design for the viewer including onboarding and context of viewing; b) the viewer’s role in the narrative; c) the spatial distribution of dramatic events within the 360-degree space.

​

Context of viewing

​

For Entangled, the author devised a backstory and a set in form of a venue, in which the audience member(s) are asked to choose between an orange or blue drink, after which a corresponding experience is presented in VR. The film is a continuation of this backstory, the viewer finds himself in the very same place, surrounded by other participants wearing VR headsets. This is an effective mechanism as VR requires the filmmaker to focus more on user experience. This process of on-boarding is priming the viewer before they enter the virtual world. It can structure anticipation and expectation in a way that increases the impact of what is presented.

The viewer’s role

​

The viewer in Entangled takes on a predominately passive role (despite controlling the gaze), a witness in the centre of events in a basement set. Represented as a disembodied observer without agency, the cast does not directly acknowledge the audience’s presence. The situation changes abruptly in the last part of the film, as the viewer is transported into an abstract realm on location in sand dunes, where an actress addresses the viewer directly – the most effective part of the film in my view. A central property of a compelling VR experience is the sense of presence a viewer might be able to experience, afforded by the immersive technology and determined by digital placemaking, user agency, proximity, and in the narrative domain, the role the viewer inhabits and how the surrounding responds to his presence. Unlike traditional cinematic representations, where montage changes the viewer’s perspective continuously, immersive experiences are typically told from a fixed and subjective point-of-view. An important consideration for the design of an immersive experience, as it lends itself to a first-person perspective narrative. To take full advantage of this subjective perspective, one needs to consider the space around the body and the proximity of people or objects, as it is of fundamental importance to how we perceive, process and interact with the world. Objects close to, or in touch with our body (personal space) are perceived with a multitude of senses and elicit enhanced neural and behavioural responses. In Entangled, the action is primarily staged in the extrapersonal space, which limits the impact the spatial narrative has on the viewer

​

Spatial distribution of dramatic events

​

Telling a story in VR has its unique challenges and opportunities and telling the story so it immerses the viewer is a central aspect. The author states, Entangled is breaking new ground by not relying on directing the viewer’s gaze, which is typically achieved by means of auditory and spatial stimuli. An interesting proposition, as it challenges the notion of the limited capacity of attention. A fundamental principle of perception is the ability to focus on some aspect of the perceivable information while filtering everything else out, also called the “the cocktail party effect.” This works well in real life for most people, however, it does not directly translate into VR. The quality of technological mediated spatial queues (such as ambisonic sound) does not always allow for efficient focusing and filtering. Rarely do we find ourselves in situations that require us to constantly adjust our gaze, turn or look behind us. It does however alter a potentially passive experience into an active one, by forcing the viewer to engage on a physical level. But the question of what the essential dramatic events are, and their spatiotemporal placement, is in my view a rather tedious exercise in Entangled. A lot of the action in the 360 space is inconsequential, but there is no way for the viewer to know. Extras walk onto set and disappear shortly after, seemingly without an objective or consequence for the story. Are they merely there to fill the space?

The central research question of Entangled is how filmmaking processes can be adopted for omni-directional narrative, with a focus on bypassing the convention of directing the gaze. To investigate this question, the author engaged in an experimental and practice-based approach, by writing, directing and producing a VR experience which is based on the very same principles.

​

I believe this is a valid approach, as much can be learned by doing and by evaluating feedback from viewers. This qualitative analysis part is essential in my view, as it provides evidence of how the outcome relates to the research question and what the findings actually are. Furthermore, the contextualisation of the project in its field is vital. One reference I would like to point out is the immersive film Eavesdrop (David Pledger and Jeffrey Shaw 2009).

​

PEER REVIEW 2

​

The VR experience, Entangled, provides a contribution to the emerging field of Cinematic VR with a research focus on script development. The non-traditional research output examines the ‘special witness’ position in ‘real-life drama’ and explores Cinematic VR as a locative/installation-based experience. Furthermore, the creative practice research developed a novel approach to scriptwriting, which features a six-column script to provide further insights for the film crew/departments including (cinematography, production design and audio department). The VR experience is composed of 360° video and ambisonic audio and thus the dramatic story approach considers three-dimensional 360° scriptwriting.

​

Entangled provides two experiences. Both place us (as observers/viewers/’witness’) in a futuristic underground quantum technology lab. The mise-en-scène, in particular, the set-design and positioning of talents (actors) are staged in a smart way considering the affordances of cinematic VR. The mise-en-scène invites viewers to explore the detail and characters that are not necessarily part of the main dialogue. As the VR experience unfolds the story makes links and draws upon references from well-known blockbuster movies (i.e., one of the lead characters chooses not the red or blue pill as in the Matrix, but a blue or orange drink.) As the story incorporates VR headsets for actors and as audience we can observe these VR experiences through screens in the lab, Entangled shifts our perspective to an interesting (and somehow reflexive) ‘meta-level’ and intertwines audiences into the story; on the screens we see a character walking through a desert, which the story will transport us to at a later stage in one of the experiences. Towards the end of the VR experience, the story drifts into an abstract motion-scape or a ‘desert’. Again, some very imaginative production approaches (i.e. to work with a shadow to indicate the positioning of a character).

​

Filming with an omnidirectional video camera means that the 180º degree rule no longer applies. The implications are key for the entire production on set and changes the setting for all departments. Entangled provides a conceptual approach to let the drama unfold in a spatialised manner around and within a particular set. The mise-en-scène, audio and set-design guide the viewer as much as the story in the drama does. In terms of storytelling or rather storyliving (Maschio 2017), Entangled speaks to the shift from the beginning, middle and end to ‘initiation’, ‘exploration’ and ‘making sense’.

​

The research statement points out that the “grammar” of Cinematic VR and 360° filmmaking yet has yet to be developed. While in 2016 “VR is where film was in 1915.” I would argue that VR is now at the end of the 1920s. Sound film was introduced in 1927 and a number of formats were established (such as documentary defined in 1928). At the current point in time, we can note a proliferation not only of content on platforms and film festivals, but also scholarly engagement in (Cinematic) VR (Tricart and Mendiburu 2017 or Bucher 2017). There are a number of approaches ranging from embodiment (Ross 2018, Popat 2016 amongst others), to narrative (such as Ryan 2016) or presence (i.e.Loomis 2016 and Morie 2017).

​

Entangled develops a mise-en-scène that is playful and demonstrates imaginative and immersive qualities.

​

​

RESEARCHER RESPONSE

​

Firstly, I would like to thank the referees and the Sightlines editorial process. It is a fabulous dynamic to have peers provide their thoughts and insights, and always ends up with a better research outcome. It is clear both referees have understood what this research and practice was all about.

​

I agree with Referee 1’s comments that there is a disconnect between the role of the viewer in the first part of the story (silent witness to events) and the role of the viewer in the Blue ending, where one of the cast members directly addresses the viewer from a number of angles. One of the most significant learning outcomes for me as a writer/filmmaker exploring this new medium was to much more clearly understand the range of formal options for the role of the viewer.

​

Since the making of Entangled, a number of researchers have explored this territory. Catricala and Eugeni (2020, 82) suggest that viewers are “immersed in a coherent indirect world they can explore, but whose narrative developments they cannot change.” Dooley (2020, 82) suggests that the viewer is “either silent witness, participant or protagonist” although Engberg and Bolter (2020, 92) remind us that the “viewer always maintains some degree of awareness of the medium and the conditions of viewing.” Nicolae (2020, 174-175) suggests that the viewer can be positioned as hero (the viewer’s presence is acknowledged by characters in the story world), witness (an invisible and unacknowledged bystander), or impersonator (the viewer is placed within the visible body of a character in the story world).  Larsen (2018, 78) suggests that narration in CVR would be better served with the viewer as sidekick to a main protagonist, so they “get to go along for the ride.” My work on Entangled led me to conclude that the viewer is always a witness, is either acknowledged or not, and can become partially visible due to rendering of body parts (e.g., you see your virtual hands or legs). With this deeper understanding, if I were to rewrite the script, I would ensure that there was a consistency in determining the role of the viewer – and I would have had characters address the viewer all the way through the film.

Referee 1 also raised a point about the “inconsequential” action. The design of the action/exposition around the viewer was meant to provide a range of clues as to what was happening and to help contribute to overall meaning-making – to clearly differentiate the narrational strategy from traditional movie-making. One of the challenges for me as a writer was finding the balance between offering overly obvious exposition (from different characters/events in different places around the viewer) but making the narrative clearly understandable. What I learned, in the end, is that – if you want to avoid directing the viewers gaze, you need to evoke a sense of meaning rather than show it. The cost of this trade-off is there is not as much obvious linear information for audiences to think about. If I were to make another VR film, I would employ much more poetic, sensory narrative strategies, and play to emotion rather than logic.

​

Referee 2 makes an interesting comment about watching the film seated and standing, and we had a long debate (with much testing) about the exact positioning of the camera, opting for placement in between standing and seated eye level. The decision was based on our tests that revealed that at standing eye level, the wide-angle of the lenses made the viewer seem like they were floating higher than other characters, and at the seated level the view felt unnaturally crushed.

​

I agree with both referees that the research statement could have included much more background and contextualisation, but with a 500-word limit this was not feasible. I would encourage the Sightlines Editorial Committee to increase the word count. For anyone interested in a deeper understanding of the practice-based research aspects to Entangled, I am hoping to shortly publish a detailed paper documenting the research journey through the making of the film. This paper includes most of the references suggested by the two referees. I have also made some changes to the Research Statement based on the feedback, strengthening the link between practice and research.

​

I will follow up Referee 2’s suggestion about VR and mise-en-scéne as it was a key observation that much of the work of directing in VR filmmaking becomes staging in the environment (as a substitute for coverage and editing), clearly signalling that mise-en-scéne has a different (enhanced?) role to play.

​

​

REFERENCES

​

Catricala, Valentoni and Eugeni, Ruggero. 2020. “Technologically Modified Self-Centred Worlds: Modes of Presence as Effects of Sense in Virtual, Augmented, Mixed and Extended Reality,” in Meaning Making in Extended Reality, edited by Biggio, Federico, Victoria Dos Santos and Gianmarco Thierry Giuliana, 63-90, Rome: Aracne Editrice.

​

Dooley, Kath. 2020. “A question of proximity: exploring a new screen grammar for 360-degree cinematic virtual reality.” Media Practice and Education 21:2, 81-96.

​

Engberg, Maria and Bolter, Jay David. 2020. “The aesthetics of reality media,” Journal of Visual Culture, 19:1, 81-95.

Larsen, Mads. 2018. “Virtual sidekick: second-person POV in narrative,” VR Journal of Screenwriting 9 (1), 73–83.

Nicolae, Dana Florentina. 2018. “Spectator Perspectives in Virtual reality Cinematography. The Witness, the hero and the Impersonator” Ekphrasis: Cinema, Cognition and Art, 20 (2), 168-180.

​

​

REVISED RESEARCH STATEMENT

​

VR is where film was in 1915

Scott Francis, Chief Technology Officer, THX – September 2016

​

Research Background

​

As a screenwriter and filmmaker I was drawn to VR when content started to become more freely available at international film festivals in 2017-2018. However, I felt constantly disappointed at the experiences on offer, which didn’t seem to be making the best use of the technology and the immersive environment it could create. In particular, VR filmmakers were constantly using “tricks” to make sure the viewer was looking where the creator required them to look at any given moment. It occurred to me that many of these new experiences could be compared to the works of early filmmakers who were exploring the new technology of film with their existing knowledge of drama from the world of theatre: the “grammar “of filmmaking had yet to be developed.

The making of Entangled – with the help of a small innovation grant from the School of Creative Industries, University of Newcastle – had two main research objectives. Firstly, to explore how the new medium could be fully used in the service of dramatic storytelling; and secondly to investigate the best way to “script” the experience, given that, while the action of a VR drama remains linear, it takes place all around the viewer, a point of difference that is not easily accommodated with traditional screenplay conventions.

​

The Experience

​

Entangled was conceived as more than just a 360 film: it is designed to take place in an easily constructed “venue”, with each audience member asked to choose between an orange and a blue drink. Once he/she has done this, a headset is provided (the orange and blue experiences are different), and the immersive experience begins. The first thing the viewer sees in the virtual world is other people choosing their drink and putting on headsets. The main drama then begins.

​

Contribution & Significance

​

Entangled breaks new ground by providing an immersive experience that does not rely upon directing the gaze of the viewer. In a sense it is more like “real life” drama: if a bomb explodes in a shopping centre while you are there, you may be facing the explosion or not. Depending on where you are, and your orientation, your experience of a dramatic event is different but – with careful design – no less powerful. Entangled finds a way to provide immersive drama using the idea of the “special witness” – who is placed in the middle of things.

​

Furthermore, the development of Entangled trialled several different approaches to script development from a professional practitioner’s perspective, discovering important insights into the kinds of tools and processes required for the development of a VR experience with ambisonic audio. A key insight is that the traditional script is still required for actors, but is of limited use for the director, production designer and cinematographer, where six column scripts were developed to identify what was happening in each of north, south, east, west, zenith and nadir sections of the space.