The works of teamLab—a Japanese digital art collective—surround viewers with universes of beautiful immersive images where physical movements trigger changes in the visual representation: with a gentle touch, a butterfly flutters and flaunts its tender dance; with human bodies drawing close to the display, waves sweep the wall of ocean painted in classical Japanese style. This is the first encounter with which many people get to know “immersive art.” It is not only an interactive entertainment but also an elaborate multisensory feast.
However, is “immersive” a word of the brand-new era? Is it a new concept in aesthetics discourse following the latest technological advancements? May it be an existing notion developing for a considerable amount of time in art history, or a trend long forecasted by current technology and information? What kinds of approaches can we adopt to see immersive art? Drawing on perspectives of film studies and theater, this article re-examines the meaning and implications of the word “immersive art,” and, in so doing, attempts to respond to its important claims. For example, how are the “sensory experiences” and “interactivity,” achieved in virtual reality films which are not possible in traditional cinema, different from that in conventional aesthetics discourse? Is it because of conceptual relevance that immersive theater, participatory theater, and environmental theater are used as interchangeable terms in the media, or simply because it is trendy to do so? Is the praise for immersive art a blind admiration commonly seen in society every time when a new technology appears, or is it grounded in the innovative changes this art form can really bring about? Is immersive art a relevant notion in “contemporary art,” “new media art,” or “digital art;” or is it just a buzzword which would soon disappear?
teamLab’s work, Black Waves: Lost, Immersed and Reborn, at the exhibition Espacio Fundación Telefónica in Madrid, Spain, in 2019. Photo by HUANG Hsiang-Yun
As with immersive cinema, the term usually refers to “virtual reality films.” When the audience puts on a VR device and enters a world of 360-degree images and surrounding sounds, it is as though one was completely immersed in another universe. Photo by HUANG Hsiang-Yun
Browsing through search results of media coverage, we see the widespread usage of “immersive art” in many fields. In terms of “mediums” as categories, the phrase can be found at least in literature, theater, cinema, as well as immersive installation in digital art. In the broadest sense, immersion refers to the experience of deep mental involvement—synonymous with the act of focusing one’s full attention—while reading a novel or watching a performance. Scholars with a research interest in literary theories, furthermore, analyze how story plots and narrative methods in literature help the readers feel absorbed. Such narrative analysis is also often utilized to approach traditional cinema and theater where people sit and watch films/performances in a black box. In comparison, “immersive theater” described in current trends emphasizes spatial and sensory immersion instead of immersion in (the narrative of) a novel through imagination and mental involvement. In immersive theater, audiences need to explore, to act, and to look around in physical theatrical spaces. One of the most renowned examples is Sleep No More, an adaptation of Shakespeare’s Macbeth. Most media coverage of the play particularly highlights the “experience” of immersive theater which cancels the distance between the audience and actors.
As with immersive cinema, the term usually refers to “virtual reality films.” When the audience puts on a VR device and enters a world of 360-degree images and surrounding sounds, it is as though one was completely absorbed into another universe, sometimes even with possibilities of interaction and haptic/olfactory stimulation. Similarly, (interactive) immersive experience in an environment in digital art contexts emphasizes full-body immersion with comprehensive multisensory stimulation like that in teamLab’s works. The difference is clear: with a VR headset, one experiences the simulation of walking in a virtual space—the virtual reality, while one can actually walk in a physical space in the exhibition of teamLab’s works where projections covering the whole place and well-crafted interactivity help the audience submerge into another universe.
When “dimensions of immersion” are under analysis, six different categories can be specified: sensory immersion, proprioceptive immersion, ludic immersion, spatial immersion, narrative immersion, and emotional immersion. 1 The dimensions are not exclusive to one another. In fact, they are not often experienced in isolation upon the perception of artworks, and are frequently used as interchangeable terms. In addition to the six dimensions mentioned above, I believe it is necessary to add “temporal immersion” to refer to temporal senses audiences can experience in a virtual reality work—spatial recognition is delicately intertwined with temporal sensibilities even when the audience’s physical body remains still. In the following paragraphs, analysis of specific artworks and what dimensions of immersion they offer will be presented.
In recent years, not only VR films are considered immersive by default, but many multi-channel video installations2 are also labeled as “immersive.” For instance, CHEANG Shu Lea’s 3x3x6, exhibited in the Taiwan Pavilion during Venice Biennale 2019, is described as an “immersive” installation3 for the reason that the audience is surrounded by multiple screens across the exhibition space, in which one room, in particular, is designed to imply the structure of the panopticon: viewers can stand in the center of encircling screens while experiencing the images of the artwork.
This form of multi-channel video installation in an encircling arrangement is considered another typical form of immersive aesthetics. It is often discussed by art critics in conjunction with the mental state of spectatorship being “absorbed” into and “alienated” from the experience. The definition of “immersion” here bears close ties to discussions in traditional film narratology and literary narratology. For example, in “The Rebellion of Moving Image” at MOCA Taipei—Rebel in Order to Return, Right to “Return,” to Get Absorbed Again,” written by WU Mu-Ching and YEN Xiao-Xiao, multi-channel video works are investigated in relation to states of being “absorbed” and “alienated,” “She (HUANG Hsiang-Ning, curator of the exhibition) brings practical details of pressing issues and discussions of aesthetic forms together. Through the emphasis on sensibilities of different dimensions in ‘The Rebellion of Moving Image’, a rebellion against a unitary official narrative is staged and the very essence of images, including sounds, is questioned. The call to return to video narrativity responds just right to the unprecedented groundbreaking motto ‘Get Absorbed Again’ proposed amidst the encasing, dissembling, and reconstruction of fictional/non-fictional narrativity as well as amidst the re-examined oscillation between absorption and alienation indicated by the exhibition title.”4
CHEANG Shu Lea, 3x3x6, 2019, multi-media installation. Photo courtesy of the artist and Taipei Fine Arts Museum; photo by LIN Guan-Ming
In response to aforementioned discussions of alienation and absorption in “The Rebellion of Moving Image” at MOCA Taipei, what’s worth mentioning is the spectator theory in the Expanded Cinema Movement in the 1960s.5 Deeply influenced by sociology, the theory argues that an audience who sits in the black box of cinema and watches films would fully accept the storytelling and the ideology conveyed by films, and, therefore, would lose the capacity of critical judgment. The form of multi-channel videos and the type of arrangement to make the audience walk between images in the space is, therefore, preferred—shifting the emphasis onto and rediscovering the value of “alienation.” A common technique to achieve the goal is the inclusion of filmmaking equipment in the pictures or the use of non-diegetic sound, in order to make the audience realize what they see is an illusion. By doing so, it prevents them from a complete “immersion” into plots and taking in ideologies of films without much thinking.6 According to the argument, the connection between immersion and illusion is even more present when it comes to VR films. In The Metaphysics of Virtual Reality, Michael Heim defines virtual reality as such: “Audio with three-dimensional acoustics can support the illusion of being submerged in a virtual world. That is, the illusion is immersion. According to this view, virtual reality means sensory immersion in a virtual environment,”7 which involves sensory and spatial immersion. On the other hand, in the context of expanded cinema, the investigation of the immersion of an audience lies more on the question of whether it is a passive or active spectatorship; whether the audience is absorbed or alienated. Looking closely into the literature of expanded cinema, we see that the word, immersive, is used carefully—only when it comes to multi-channel video, surround-screen film, and spherical projection. In Mandarin Chinese, however, immersion can mean the act of focusing one’s full attention without necessarily referring to the all-surrounding illusion, the encasing of an immersive environment, or perceptual empirical incidents formed through interactions between a body and images.8 The default usage of immersion in many discussions about VR films without nuanced arguments is nothing but a confusion which results in art critics in the Mandarin-speaking world commonly misusing the phrase, immersive aesthetics.
The installation view of Isaac Julien’s Ten Thousand Waves in The Rebellion of Moving Image. Photo courtesy of MOCA Taipei
In terms of the developments of media art, “the narrative of a film” and “a dark environment” make viewers in cinema feel deeply “absorbed” into the plots while watching films, whereas multi-channel video or surround-screen installation in art museums “alienate” the audience from the contents by making them walk— the audience has to walk, to make active choices of where to look at, to stay busy and keep moving around. Interestingly, a sense of spatial and sensory immersion arises when one is moving in a physical environment surrounded by multisensory stimulation.
When developments of traditional cinema have catapulted us into the new era of VR films, how do we re-consider the definition of cinema and the implications of immersion? Movie director, Steven Spielberg, didn’t like virtual reality when he first encountered the technology. “I think we’re moving into a dangerous medium with virtual reality,” he said, “The only reason I say it is dangerous is because it gives the viewer a lot of latitude not to take direction from the storytellers but make their own choices of where to look.” And this takes away the storytellers’ power to dominate viewpoints of the viewers.9 Movie director TSAI Ming-Liang also said: “The nature of VR is intriguing. It is created purely for an audience. The audience can choose to look at the scenes. They can choose to look at the actors too. As a result, it also poses a big challenge for the actors. For example, the leading actress in the film, YIN Shin, considers it more difficult to perform for VR films. There is not a fixed camera for actors to look at. She/he has to make herself/himself an integral part of the environment. In a way, it is more like performing in the theatre.”10 If we frame it with a more theoretical approach, we can take film researcher SING Song-Yong’s explanation as a reference point: “As with mise-en-scène in filmmaking, instead of terming it as lack of an absolute camera angle in the shots, we can say it is only left for (the subjective viewpoint of) a post-audience to choose with great freedom where to look at.”11
SHIH Wei-Chu, a scholar with a research focus on film history and theories, compares the frame of traditional film narratology and the visual field of VR films in Thoughts on Film Terminology and VR Movies. In traditional films, the frame draws boundaries for viewpoints of the narrative. The distinction between off-screen and on-screen is, too, an important reference axis with which filmmakers create stories. In comparison, there is no limitation of the frame border in VR films because it is a 360-degree cinema.12
SHIH Wei-Chu argues that even though there are different limitations on the medium itself of both traditional films and VR films, it is up to directors and screenwriters to decide whether the concept of the frame is adopted, or whether certain contents go into the “frame” (it is not a dichotomy where content is positioned against form, but here we divide the two for now). SHIH uses French director Alexandre Perez’s VR film, Sergent James, as an example. Starting with the subjective point of view of the main character hiding underneath a bed and looking outward, the film still draws on the traditional concept of the frame. In comparison, LAU Kek-Huat’s VR film, Feathered Dream, adopts a bird’s-eye view for the audience to observe daily lives of the characters from above while being immersed in a dream-like floating state.13
Upon examining VR films, scholars and movie directors mentioned above who have long immersed themselves in movie studies tend to address the tension between the audience’s autonomy to choose narratological viewpoints and the director’s power to dominate the narrative. Contrary to traditional films, the freedom to make choices of images and the interactivity—characteristic of the VR medium—can be contradictory to immersion in the sense of “the act of focusing one’s full attention” under certain circumstances. On the other hand, in the digital era where interactivity is more than ubiquitous, interactivity as a new methodology of storytelling may not necessarily go against the very idea of immersion itself. That being the case, the actual content of sensory stimulation and details of the implementation of interactive design is then the center of attention in future research of immersive aesthetics.
A still photo of Afterimage for Tomorrow. Dancer: PAN Bo-Ling and FANG Yu-Ting. Photo courtesy of Kaohsiung Film Archive, photographer: LI Meng-Ting
Responsive to the discussed issues with approaches utilized in body phenomenology theories, philosophy scholar GONG Jow-Jiun addresses how interactive design of VR technology affects the extent of immersion that an audience experiences. In That Fire, That Cloud of Smoke, Immersion of Anti-Immersion: ‘Afterimage for Tomorrow’ and VR Film’s Manipulation of Proprioception, GONG argues that the most important concern in VR film narratology is to create different sensory situations while at the same time making the audience feel absorbed in the narrative and stay immersed in it.14 “With the help of headsets, what awaits viewers are certain immersive situations. How the proprioceptive focusing of senses of immersion and presence can be maintained when viewers turn around physically in the space with headsets on (so that viewers can psychologically and emotionally transition into certain situations and accomplish situational shifts of proprioceptive focusing with different physical modes in various scenes), then, lays the foundation of crucial operation and basic sensory logic for VR films to evaluate whether the sense of immersion is disrupted and whether it is causing the audience to feel alienated from the experience.”15
In the article, GONG Jow-Jiun proposes a presumed aesthetical criterion. Interactivity may go against cinematicity because redundant or ill-designed interaction alienates the audience and results in a lack of cinema qualities of the experience. But GONG specifically puts the word “cinematicity” in quotation marks to highlight that it involves stretching the definition of cinema itself. It also leads us to ponder: could the emergence of VR technology urge us to establish a new aesthetics of film? Could the aesthetics be based on the expansion of immersive “interactivity” and “sensory experiences” instead of underscoring the immersion made possible by the narratological principles of traditional cinema?
In addition to immersive films, many contemporary theatrical pieces are also labeled as immersive. Art critics almost use immersive theater, environmental theater, site-specific performance, and participatory theater as interchangeable terms. Although these forms of performance require participation from an audience, allow actors to walk among the audience, and break the boundaries of the stage, the fact that they are all categorized as immersive theater simply because they redefine the stage boundary or because they are performed outdoors instead of in the black box is deeply problematic. The categorization of various works should be conducted with comprehensive consideration of their contents, at which time they are created, and further relevant contexts.
Site-specific performance appeared firstly in contemporary art theories in the 1960s. In contrast to concepts such as space and place, the word, site, generally connotes a more concrete sense of a location, a spot, or a site where things happen. One of the contexts behind site-specificity is theories about minimalist sculpture which emphasize the perception of an artwork by an audience in a specific environment over the artist who created it.16 If the artwork is removed from the particular site, it is then fundamentally changed or destroyed.17 In this sense, the participation of the audience is a priori for the completion of work.
When site-specificity is applied to theatrical works, the site of the performance and its relation to the narrative are underscored. A theatrical piece must be accomplished in a certain environment to be completed. If the designated environment of the performance is altered, the piece is incomplete. The definition of environment in environmental theater includes a broad variety of elements—the avenue, props, ticket office, toilet, lighting, costumes, the actors themselves, and all which are decided by a theater group. That is to say, everything situated in a particular setting that would play a part in affecting the audience’s experiences is seen as part of the “environment.”18 Discussion of environmental theater also sees its relevance to happenings, an earlier movement in art history. After the 1970s, an ecological connotation is added to the discussion19 when it comes to the usage of the word, environment. Since environmental theater foregrounds the idea that every element in the theater is at an equal footing to one another, has its own language, and voices out on its own behalf, the audience, then, can be the leading character of a play, so can the stage or props—they share the same significance with the actors.20 Drawing further on concepts related to ecological sustainability, environmental theater becomes a place where all humans and non-humans, as equals, are in dialogue with one another, which can also be seen as a manifestation of intersubjectivity.
Such pursuit for intersubjectivity and for creating platforms to have dialogues with the Other coincidentally matches the ideals of participatory theater. The most renowned discourse of participatory theater concerns how it can represent or have direct conversations with those who are oppressed. Unlike the theoretically assumed audience in site-specific art which is often just an abstract idea, or the middle-class audience who goes regularly to outdoor art festivals—defined under constraints of certain ideology in specific conditions, theater of the oppressed stresses the link to social reality and what happens with real human beings. It is not a disrupting accident that environmental theater is later associated with ecological and posthuman lexicons, as well as theater of the oppressed that relies on sociological perspectives.
In his book Environmental Theater, Richard Schechner proposes a theater spectrum as a tool to analyze durational events, from public events which sit close to the general idea of life, to what the public considers “pure” fine art, such as performances in theaters.21
From “impure” life to “pure” art:22
Image by HUANG Hsiang-Yun
In what sense, however, is immersive theater related to previous theater genres? Current discussions about immersive theater are still under development and cross-disciplinary endeavors which include theater studies, film studies, and new media theories haven’t yet come into fruition as a systematic framework. From what I have gathered so far during the early stage of research, I believe we have to go back to the aforementioned six types of immersion, and to understand that immersion is, too, a spectrum. That is, it is a question of “extent of immersion.” Based on such understanding, nuanced differences can be further analyzed according to the various dimensions of immersion, including narrative immersion, sensory immersion, interactivity, and emotional immersion. In my opinion, it is more appropriate to only refer to theater pieces with a specific focus on sensory, spatial, and temporal immersion as immersive. For others, “environmental theater” may be a better description given corresponding consideration of the contents and stories, instead of simply using the word immersive to blanket all. As with performances in the form of games with rules and which allows the audience to walk around and explore, ludic immersion can be a suitable term followed by enquiries into another aspect. Different types of immersion can happen in the same work. The categorization, after all, is a labeling structure for reference. Extensive analysis of contents and forms is more significant because immersive aesthetics depends largely on contexts and audiences due to the nature of the mediums. There is not an intrinsic quintessence of immersive art that rings true forever. There is not an inevitable quality only achievable by certain technology, either. The notion, immersion, is eventually a buzzword lest it brings new understanding and interpretations to approaching works. The ensuing research of this article will keep its focus on interactive narrativity, perception, and new media. With specific artworks as departure points, the endeavor to clarify the concept of immersivity or to look for new terms that are more accurate will be continued in order to truly find the foundation of understanding contemporary immersive art.
