top of page
Research and Critical Reflection 3

In this critical reflection, I will take the body inside the virtual environment into the focus to discuss different approaches to visualise the experience of existing in a mixed-reality. I will explore how this overlap of our physical and digital modes of existing in a body can be communicated through the creation of mutant spaces in painting and virtual environments of the in-between. I will continue to evaluate the use of new technologies combined with traditional painting by setting a focus on the innovative potential of VR and AR. I will end this section of my critical reflection with a thought experiment that connects the glitch to the force of gravity and give an additional entry point to how we can understand the mass of a body from a philosophical point of view. In the second part of this critical reflection I will reflect on my most recent work and evaluate how it expresses and translates my body of research.

How can art visualise the feeling of existing in a world in which interactions or experiences can no longer be perceived as linear?

The construction of perspective in painting has evolved steadily over the centuries, with breakthroughs in different parts of the world bringing forth new approaches to depicting our perceived reality and responding to the particular cultural needs of the times.

 

When looking at medieval painting, the representation of religious experience was paramount to that of human life, thus the importance of holy figures was easily perceivable by their size in relation to other less relevant figures within the pictorial space. A similar hierarchy can also be found in early Egyptian art, where the important figures were also highest in composition, leading to the so-called "vertical perspective". The approach taken in Oriental art differed as spatial depth was attained by an overlap and what might be called "planar" perspective, consisting essentially of distributing compositional elements on spatial planes. ”Earthly bound" objects (people, animals, buildings and forests) belonged to the foreground and "heavenly" elements (hills, mountains and sky) to the background, each gradating in hue, detail and tone. Painted onto big formats these compositions were constructed to directly relate to the body, so that the viewer could walk along the work and immerse himself in the experience of nature that radiated from the work. After the birth of one-point-perspective, the viewer was first able to sense space almost fiscally. A truly geometrical based perspective was developed in the Italian Renaissance and its development spans over the fourteenth and fifteenth centuries. This exploration was based on the mathematical form of Alberti which was based on the belief that no picture could “resemble nature unless it is seen from a definite distance and location, and the diminution in size as a function of distance” (essentialvermeer). While the environment could from then on be constructed, the perspective of individual objects or figures was generally omitted from the procedure. "Artists could construct the perspective grid that defines the stage and the location on the stage of the actors and props, but they did not explicitly develop the images of objects using strict perspective methods” (MacEvoy, 2015). Two-point perspective was used successfully for the first time in 1650 by the Dutch Artists Gerard Houckgeest and inspired artists worldwide to replace their use of one-point-perspective with two-point perspective to create a greater sense of compositional dynamism, widening and accentuating the illusion of reality so that the depicted reality was so convincing that the viewer could almost step into the scene unfolding before his eyes. (essentialvermeer)

 

What these advances in reconstructing the feeling of entering a space have in common is that they strive to represent and capture fictional or historical events and to make them easily readable and convincing to the viewer's eye, while accommodating his understanding of space. They all follow the rules of Euclidian geometry, which describes a space that is flat whereas non-Euclidean space is hyperbolic and connected to Einsteins theory of general relativity and space-time. This progress in Maths and Science opened up a new way to describe the universe which also found its way into the arts.

“Animated by a new conception of the world, the arts in a collective fermentation have begun to stir. And each of them has evolved with a new dimension. Each of them has found a form of expression inherent in the next higher dimension, objectifying the weighty spiritual consequences of this fundamental change”, Charles Sirato writes in Manifeste Dimensioniste (1936). He proposed that each art form should start to expand into an addition dimension, meaning for painting to “leave the plane and occupy space” (Sitaro, 1936). This appeal was largely followed by the Surrealists around Breton who began to create new spaces, as for them non-Euclidean geometry signified a new freedom from the tyranny of the established. When Poincare published his philosophy of a relativity of knowledge painters started to explore the 4th dimension. Non-Euclidian perspective was used to express something that goes beyond what can be seen, sometimes even something beyond what can be imagined. (Naiadseye, 2014) Spatial illusion was almost banished from modern painting with the Minimalist movement of the 1960s. Tony Robbin negates this by stating that “art goes beyond mathematics or physics. Artists who are interested in four dimensional space are not motivated by a desire to illustrate new physical theories, nor a desire to solve mathematical problems. [They] are motivated by a desire to complete a subjective experience by inventing new aesthetic and conceptual capabilities. Our reading of history of culture has shown us that in the development of new metaphors for space, artists, physicsts, and mathematicians are usually in step.” (Tony Robbin in Naiadseye, 2014)

I would like to explore other ways of structuring space to create the sense of a collision or a rupture of our reality caused by the additional digital environment that is becoming more and more intertwined with our physical everyday life. In the following section of this reflection, I will compare and discuss different artistic approaches to inventing a new shape for the body in this space and reflect on the means by which they make visible the invisible in their life. I will explore different approaches to create a balanced composition of two juxtaposed realities.

 

In the works of Surrealist Roberto Matta, the collision of living and non-living matter is explored, with a focus on cosmic creation and destruction. His early works resemble interior landscapes of “indefinable geometries and sensuality, a permeable looking glass into the psyche’s appearance, where time and space occur simultaneously” (Rosa, 2016). He created a unique and intuitive process to make the invisible visible and to give a glimpse into the fantastical and inward-looking narratives of the distant universe of his mind. To achieve this effect he crossed linear-perspective with that of hyperbolic space to create the sense of a mixed reality. His approach can be compared to that of contemporary artist Maggie Roberts, who aims to give shape to a post-anthropocene world where life forms inhabit a multidimensional world, “where the virtual bleed[s] into a post-apocalyptic everyday” (IMT). She expresses a cracking and leaking of our categories and boundaries, that cannot take the immense pressure they are under. This leaking gives way to “a fluid and complex uncertainty populated by entities on a continuum between the human and the non-human, a mix of the organic, the engineered and the synthetic.” (Roberts, 2021)

How does the body take form and how does it re-shape in this hybrid environment?

Hsus approach to visualise the effect of digital technology on the way we perceive our body and our environment is almost paradoxical as he brings the body into a space that is formed by the synthetic, the technological and the artificial but his use of material remains completely organic. He sets his aesthetics apart from the ‘cyberpunk’ aesthetics of the early 1990s in negating the focus on the immaterial mind and the doing ways with the body. Hsu insists on the fundamental corporeality of our encounter with such virtual systems, because he believes that the body does not act as a disposable prosthetic, but as “a kind of interface, a place that connects various systems of reality” (Jeppe Ugelvig in Miguel Abreu Gallery).

He is recording technology’s integration with the body, probing the cognitive as well as physical effects of transformative technological advances on our lives. His work reflects his early assessment that technology was becoming an extension of the human body, which is a condition he concluded was destined to intensify over time. His sculptures contain elements that contain bits of digital data, while hinting at contraptions yet to come. Hsu uses innovative fabrication techniques and materials – particularly silicone and alkyd, a durable synthetic resin – to call to mind bodily orifices, organic matter, or biotic growths. At the root of these works is the question of technology’s effects – whether distorting, surveilling, or life-giving – on human beings. (Julie Belcove in Miguel Abreu Gallery)

He explores a new way to talk about the body that does not necessarily refer to past imaging of it and instead examining his own experience in the world. (Hsu, 2022) In his practice he aims to “get into the synthetic, the technological and the artificial but at the same time to bring in the body as well” (Hsu, 2022).

 

This begs the question whether the use of new technology is necessary to talk about technology, or whether a purely painterly approach can express ideas and change equally well?


Hsu explains this disregard for new technology in his process with his lack of interest in making art using technology as a new medium, as he perceives the current situation as much more complex. For him, not only is the world becoming more technological, but the media that artists can use are also changing. Hsu insists on staying within the medium of painting, which has been continuously used by artistic movements of the past to express differences in our way of life over time. In his view, artworks created with new technologies lose this connection to the ongoing discourse of the past and therefore cannot express change as radically. His focus is not on the use of new technologies, but on recording how the way we relate to them is changing. (Hsu, 2022)

I believe in the potential of painting as a medium to visualise our contemporary experience in terms of progressive digital immersion, but I think it is necessary to connect the painted work to a virtual or digital dimension as well, given that the way we approach works of art is changing as well. I feel that through the use of new technologies, the artist can more honestly experience the sense of immersion and the effects of digital space on their physical space, and therefore can use or critique it from a point of internalised learning.

 

Digital artist Jacolby Satterwhite approaches the quest for true embodiment from a perspective entirely opposed to Hsu's critical perspective. Working towards healing his body internally and externally from ruptures he experiences in the physical world, Satterwhite disintegrates his body into a network of digital worlds that virtually emerge and are embedded in his own joints. In his video work “Pygmalion’s Ugly Season” the viewer enters into his private world that he constructs from within himself. What he finds “are cloaked meditations on desire, human nature, healing and finding utopia” (Lambo, 2022). Satterwhite expresses a desire for unattainable perfection that implodes under its own pressure and gives way for a confrontation with a multitude of utopias. He is creating the landscape he yarns for and populates it with people close to him that are appearing to be fully immersed it this new environment. This Virtual Reality becomes a catalyst to communicate what is feels like to live in his body to an outside. It creates the ultimate transparency. Seemingly subjective, it is based on peoples responses to the question of what utopia would look like to them. Their responses of consumerism, nature, infrastructure, domesticity, religion and philosophy are matched with the floating spherical worlds that the viewer enters into.

But is this an accurate representation of their individual visions of the future? Is an environment created in a virtual reality ultimately subjective?

 

According to Hito Steyerl, the viewer of a virtual reality is fully immersed into something that he is not part of. She understands virtual worlds as constructed bubbles in which the viewer is at the center of a sphere, yet at the same time his body is actually missing from it. Steyerl calls this phenomenon bubble vision, in which the body of the viewer is exchanged for a blind spot in the center, which means that he is missing from the scene even though he is placed at its center. For her this symbolises the fate of humanity as she connects the idea of the human disappearing from the center of this creation to a disappearance of the idea of an antrophocene. (Steyerl, 2022)

According to Korean artist Yunchul Kim the present world is in the middle of a paradigm shift, where one era ends and a new one begins, like the ends and the beginnings of a continuous spiral. He perceives humanity at “the “swollen” boundaries of complex political, cultural, social, and existential entanglements, accelerated by the aftermath of the pandemic, human-machine dilemmas, ecological disasters and warfare – all a shadow of [its] anthropocentric condition” (e-flux, 2022). At the periphery of these tensions he views the world through the lens of materials, where objects, beings, and nature all co-exist on an equal footing.

The anthropocene defines the desire of mankind to place itself at the center of the universe,  above all forms of life, in control of every development on earth and ready to irrevocably shape it to its liking. Bubble vision in VR could be used as an analogy to mankind consistently thriving for a possibility to hand over their power “to opaque automated procedures, to black box algorithms, all sorts of crystal ball gazing” (Steyerl, 2022) as soon as they are aware of possessing this power. Steyerl goes even further and assesses the potential of “worldcreation” within these bubbles. She defines the qualities of such an artificial sphere as very limited. Firstly it can only reflect the likeness of its creator, meaning it will include all of his biases or the biases of society that are being imposed on this creator. Secondly they consist of nothing but yourself and “you are not only missing [from it] but you are also on your own” (Steyerl, 2022).

 

I must admit that I share this view with Steyerl. At this point I don't feel it is possible to share VR experiences enough - I exclude gaming from this statement. I admire the openness with which the artist expresses new utopias or protopias in VR, but I find it necessary to bring these ideas and desires into the physical world, be it in the form of painting or AR. I think the viewer needs to be able to remain in their reality in order to be receptive to new concepts and transfer them into their world, rather than conceiving them as mere realms of fantasy. At the Venice Biennale this year I encountered a VR work in which I felt fully immersed for the first time. It was the work by Loukia Alavanou called “Oedipus in Search of Colonus”. He restaged this ancient drama to reflect contemporary events in a 15-minute VR film curated by Heinz Peter Schwerfel. What was very striking is that the VR experience was recored at a real site and inhabited by real people (amateur actors), who live under very harsh circumstances, lack any form of citizenship, and are being disregarded by the Greek authorities. I felt very moved afterwards and I think that VR can also immerse the viewer into real environments to realise certain realities that must be improved or even ecological sites that need to be preserved. Maybe it could accelerate the general awareness for such issues in society, even if one is just entering these spaces within ones virtual body.

 

 

What would happen if a replica of the body is inserted into VR and has the power to affect the virtual scene?

 

The architect and digital designer Dor Cohen is fascinated with the moving body in space and time, and how related phenomena translate to virtuality. In his practice he interlaces the physical and the digital in thought-provoking films, illustrations, and virtual reality experiences. (RCA, 2022) In Dors video work “Estrangement”, he himself enters a virtual scene that is almost completely vacant except of his body that is being mirrored at a distance. In this scene Cohen possesses the power over his body and can move freely around the space. With every move towards his mirrored second body, its form changes in a deformation that is distorted synchronously to his movements. The digital body becomes a mark. As he is gaining control over the other body he unlocks a limitless array of new moves. He explores the body as a tool to explore and develop movement and aims to exceed its natural limitations and capabilities to enrich our experience in the world. His work challenges the notion of embodiment and opens up new possibilities to think about our own disembodiment. (Cohen)

I relate Cohens dancing and shapeshifting bodies to Satterwhite's video work, in which two bodies can been seen dancing and reacting to one another in VR. Both bodies seemingly want to connect and intertwine. For me, this expresses the striving for more tangible community in the digital space. Both visualise the dream of actually coming together in a virtual but also a more embodied way. The fusion of bodies can also be observed as a central element in Kumbirai Makumbe's work. Their work "Living Doesn't Mean You're Alive” from 2021 questions the body as a site of emotional emancipation along with the notion of reaching a technological singularity. They continuously question the multidimensionality of blackness, queerness, transcendence and “in-betweenness". Drawn to the materiality and malleability of digital matter and the infinite possibilities of its employability, they transform the body to explore future ways of being in a body. (arebyte, 2022) Makumbe explore that what is lost of one’s physicality during a transformation towards progress. “It is a moment of questioning what truly fuelled their yearning to emancipate themselves from the confines of their biological human body. What is the body itself? or what is the experience of inhabiting their body in the context it existed within?” (arebyte, 2022) Their work has a very soft formal language, wafting organic matter is combined with amorphous figures that are held in a common fusion, sighingly circling on floating plateaus. The human has emancipated himself from his body and has ascended to an “informorph”;  a virtual body of information that possesses self-awareness and scentience” (arebyte, 2022). Makumbe pose the question of what could be gained and what could be lost if humanity succeeds in separating mind from body, and compels us to rethink, if that is even a desirable outcome.

Hsu sees us as unable to answer this question in the here and now. According to him, all that can be said is that being human means being in the world and nothing more. We can only comprehend the here and now, and the generations to come who will actually inhabit the future may already have a very different understanding of what their world holds in store for them and how they want to live in it. While we may know what they have lost, they will not be aware of it because they have never experienced those aspects. (Hsu, 2022)

According to Hsu, we are already seeing this shift in the younger generations who grew up with smartphones and the constant connection to the Internet. They already live in a different world where they feel no inhibitions or concerns about the technologies and devices they use. Hsu derives this from man's ability to quickly adapt to his environment, be it better or worse than that of the previous generation. (Hsu, 2022)

I agree with Hsu that the way people interact with new technologies, and especially in the digital space, is very much related to their age. There is a certain reluctance and skepticism among the older generation. The transparency increasingly sought by younger generations is rejected by the older ones, who may be more aware of potential risks, external manipulative forces, and profit-maximising entities advancing on their patterns of behaviour and information. However, I believe that it is of paramount importance not to lose the connection between both generations, in order to facilitate a progression in how we combine the physical and the digital experience in the future, fueled by the imagination and curiosity of younger generations and stabilised and reflected from the point of view of a different lived experience, which may contain values ​​and truths that would otherwise be forgotten. “It is very scary what can happen here,” says Hsu. “The power to manipulate is very frightening. Wether we can contain that power is a huge question.” (Hsu, 2022)

New media artist Pascal Sender is one of the first artists to aim for this bridge between generations. He poses the question if there is a way to paint that is native to the 21st century and also being able to relate to historic works?

What makes the works of Pascal Sender a unique reflection of modern life is the innovation that he has created so that the pictures can be viewed in Augmented Reality. He has hand coded an app, so that when viewing his paintings through your phone camera, another digital perspective is revealed. (Smith, 2020) His paintings seem to jump off the canvas appearing as three-dimensional forms, moving and interacting with the viewer. “His work never settles, it’s a movable feast of dancing lines, percussive and unapologetic. It can’t be passively viewed, it has to be experienced” (Smith, 2020). It is practically impossible to view his work or half engaged. According to Smith the viewer has “somehow stopped being able to absorb, preferring the definitive record of an iPhone photograph to the data allowance of our own brains. We shoot to remember, rewarding ourselves with a digital record of having been somewhere or experienced something.” The flood of images we collect but never fully perceive is diluting and devaluing each image. Ironically Sender depicts the human figure in even this moment, glaring at the screen of their smartphones, moving trough life in a state of distraction and unawareness. Sender questions the modern world and our need to capture and categorise. He creates work that cannot be formatted to the oblongs of our phones, by using exactly this device to commandeer our screens just to keep us focussed on his work in the now. (Smith, 2020) Sender recalls the notion of two different levels of engagement with his work. Whereas people entered the gallery to simply look at the painting, their “son or daughter had the phone in their hand and they were looking through the phone at the painting because it was natural or they wanted that” (Sender, 2022). He feels that it is getting increasingly harder to give an entry point and to let people in. While the middle-aged viewer sees the AR addition simply as an add-on, a child would not see it in that way at all. It would actually see the work in that moment, because it is able to interact and play with it. “[Children] want an interactive moment, they want to engage with it with clicking, and something happens, and it changed. And an older generation maybe exactly doesn´t want that. I find it super interesting to make a bridge between these two areas, or the two layers of how people consume art”, Sender explains.

The artist Niall Ashley explores this virtual extension of his paintings through complex performances that take place in a dual environment that is shifting back and forth between a digital and a physical world and between his digital and his physical body. He switches between the two modes of existing with ease and thereby inspires the viewer to also search for this new balance in his own way of existing in the in-between of our contemporary existence. (Gordon, 2021)

Whereas Senders and Ashleys works come to life within a new dimension of the painting, Ry David Bradley approaches painting in the 21st century from another point of entry. He paints inside the image itself, on its own terms, which is something that was impossible in anything but this century. He depicts the human face today, acting as the last form of privacy and security, but also monitoring. For Bradley the relationship between the smartphone and the face is symbiotic. The face is one place where “changes can be reflected, and so by using custom-created software brushes that leave loosely evident their fractured character, this classical trope of portrait painting can also capture the highly edited present. And whatever overlays it will continue to be adorned with as mixed reality becomes more prevalent.” (Bradley in Mills, 2022) How can our reality be understood in an alternative way and where can we find the first symptoms of this mix of elements and states? The next paragraph dives into Deluzes perception of the world and how matter can be approached from a philosophical point of view. 

What does it mean to live in the world?

Deleuze approaches this question by tying monadic thinking to the art of “displacement and transformation” (Deleuze, 1993, p.15) that is expressed in a constant flux between deterritorialisation and reterritorialisation. For him, artists are emigrant thinkers who deterritorialise accepted notions of space, aiming to “shift of the opposition of organic and inorganic matter into tonal flow and flux” (Deleuze, 1993, p.15). Through this they could also express how the two worlds - the private space and the teeming public world - melt into each other to create a movement.

To support his view, Deleuze refers to Leibniz, philosopher and mathematician of the pleat and curved and twisted surfaces, who rethought the phenomenon of “point of view”, of perspective and city planning. In his study of Leibniz Deleuze implies that an “extraordinarily delicate filigree of concepts, winding through organic and inorganic worlds, has to be retrived. (…) such that we can imagine movement of light and sound, together, as folds of ethereal matter that waft and waver” (Deleuze, 1993, p.13).

Deleuze picks up on Einsteins theory of general relativity. He understands the relations between matter in the universe as being caused by the infinite division of matter that creates a compressive force “to return all portions of mat­ter to the surrounding areas, to the neighbouring parts that bathe and penetrate the given body, and that determine its curvature” (Deleuze, 1993, p.5) The only matter which cannot be separated into parts is the absolute fluid, the passive and abstract matter. (Deleuze, 1993, p.5)

As a body has a degree of hardness as well as a degree of fluidity, Deleuze defines it as essentially elastic. He spins this thought further by stating that the body is therefore never an absolute. Its flexible folds are not separated into parts of pans but are rather divided to infinity in smaller and smaller folds that always retain a certain cohesion. (Deleuze, 1993, p.6)

Deleuze perceives the human body as a machine that consists of an infinity of smaller machines and that is only "transformed by different folds that it re­ceives” (Deleuze, 1993, p.8). Proceeding from this thought he concludes that mechanical processes can never be sufficient to be machines (in this context referring to a living body) because their internal number of reactive folds cannot match the number of folds in an organic organism. I connect this thought to Al-Khalilis statement that a programmed and artificial being can never relate to or learn from the experience of being in a body and can therefor never match the abilities of a truly sentient life form. “The organism is de­fined by its ability to fold its own parts and to unfold them, not to infinity, but to a degree of development assigned to each species” (Deleuze, 1993, p.8).

In the Barock the soul entertains a complex relation with the body. It is inseparable from the body and “tangled in the pleats of matter” but at the same time able to rise up into another sphere of being which “make[s] it ascend over all other folds.” (Deleuze, 1993, p.11)

For Deleuze the Baroque is a mystical and mathematical dimension, which may not have an empirical or historical basis, but it happens to be the virtual sensation of a moment of totalisation and dispersion [an epiphany]. It includes the thoughts of infinity that come with the view of the world in which all of its visible objects are moving aggregates of infinite numbers of atoms and molucules" (Deleuze, 1993, p.12). It its multiplicity Deleuze locates a unity which is the unity of that multiple piece, as in all of those fragments. (Deleuze, 1993, p.14) With its cavern­ous shapes a vortex is created that holds all those fragments together. For me this connects to our understanding of space-time. I wonder if a mixed reality could also be expressed visually by not only one gravitation force affecting the mass and shape of a body but two. How would those fields reshape the boundaries of the body and would it alter our way of inhabiting it?

Reflections on my artworks
How can a painting be expanded into the virtual and physical at the same time?
transformation-1.0.jpg
Transformation 1.0

Even before I included the AR component in my work, I had the vision of making the viewer feel like they could immerse themselves in the work in front of their eyes and almost enter into it. I wanted to create a smooth transition between two realities, a kind of portal through which the viewer enters a new dimension in which his body is deformed under new laws. The work is intended to be both inviting and deterrent, leaving him in the dark as to whether the developments before his eyes will have positive or negative consequences for the bodies within. The life-size figures appear unapproachable, cool but still alive. They are undisguised but not in a sexual way but represent the concept of the body being renewed at its core. The characters are in different relationships to each other and are in the process of connecting their bodies borders to form one single entity.

Viewed from top to bottom, they become more and more disembodied. They lose their organic attributes and take on cyborg-like traits. The body is deconstructed and reconstructed, but not with the human image in mind. However, viewed from the bottom up, inorganic matter is breathed life into. Technology becomes sentient and frees itself from its limiting connections to a purely digital space. It was important to me to let the development run in both directions in order to clarify the limbo in which we find ourselves.

The figures move towards a bright, flooding light that allows no information whatsoever about the new environment it conceals. It is a space in a vacuum made of pure energy. The system draws this energy from the body entering it until it has completely absorbed it. Is it a liberation from the limitations and burdens associated with our experience in organic body, or a loss of one's presence in the physical world? Only one figure is aware of the viewer's presence and the development that is taking hold of him and fixes him with his gaze. It is a sober look that appears to know but does not share that knowledge. The character looks back into the organic world and at the same time is unable to step out of the frame as its body has already lost any human traits.

The AR extension makes it even clearer that the viewer is already in the middle of this process. He is not aware of it, since this environment only becomes visible when he connects to the Internet and opens access to this new space through his smartphone.

The AR composition

My concept for the AR extension was to give the visitor the feeling that the painting extends into real space and that they too are part of this scene. In the “process” section of this unit, the entire process and my considerations can be understood. The scenery is based on the formal language of the painting and takes up elements from its composition, such as the metal fragments that remain in suspension to the right and left or the tubes that seem to grow to of artificial soil. The AR scene appears organic and technological at the same time, supporting the impression of the work as a whole. The texture of the painting is reflected on the AR surface and is deformed by the curves of the structure. The black plane on which the figures are placed seems to have flowed out of the work and virtually extends across the gallery floor. The viewer is invited to step into the scene as a metallic hand reaches out from its surface as if to grab his hand and pull him inside.

I consciously decided to implement my idea in AR and not in VR, because I don't want to remove the viewer from his reality but want to enrich it with a new dimension. I would like to visualize a simultaneity of both realities and bring them into balance, through which both levels enrich and visually charge each other. In AR the experience can be shared with others in real time and the environment is not left but included. I see an advantage in being able to activate the AR with your own smartphone. We're viewing more and more events primarily through this screen, immediately capturing moments or impressions to revisit them at a later point in time, jumping from one visual stimulus to the next at the actual point in time without really critically engaging with content. Of course, it would have been ideal to start the AR by simply registering the painting with the camera of the phone, but at the moment I have only found models that are based on a QR code or are triggered in a separate application that must be installed first.

Screenshot 2022-11-12 at 12.48.08.png
Dor Cohen
750x750bb.jpeg
Pascal Sender
58c2f07e17454dd80381f744d86b6f4f.jpeg
Kay Sage
tishan_hsu_biennale_arte_2022_600x600_0.jpeg
Tishan Hsu
The Nebula. 1939 _ MoMA.jpeg
Raoul Ubac
dsc_9794.webp
Margherite Humeau
Stylistic improvements

 

I think this work is moving in a direction that I want to continue in the future. I am particularly fascinated by the combination of painting and AR and I think that it is a suitable medium for aptly communicating the topics and thoughts of my research. With regard to the composition of my painting, I would like to be even more free in my forms and fill the format more casually. Precisely in this clash of two realities, it seems important to me to define a new approach that determines the distortion of the elements. I am wondering if the glitch could occur as a result of the clash of two different gravitational fields that are overlapping.

Cohen's video works and Ubac's solarisations provide a clue as to how the body can be interpreted detached from its actual mass. In my work I have placed a lot of value on the proportions and three-dimensionality of the body and now I think that I would like to try to combine three-dimensional and two-dimensional bodies on one pictorial plane.

Compared to Sender's AR works, I wonder if it might make sense for future works to first determine the digital 3D objects and then plan them into my painting instead of trying to render objects afterwards, the areas in the work or the brushwork could empathise with. But I'm convinced that I don't want to limit my AR elements to the surface of the painting, but rather expand its surroundings.

Humeau and Sage again motivate me to leave more “emptiness” in my composition and deliberately balance the curved elements with geometric shapes similar to the approach Matta has taken in the past. Humeau's process of research and her many sketches open the space to think about alternative elements that exist in digital space and can be transferred to the body, apart from the materiality of metal or robotics. The impulse to think more about the materiality of my works comes from Hsu's work. His experimentation with textures and different surfaces opens up a new space for me to further explore and reflect.

Technical issues

I see several future improvements in the way I can deliver an AR extension to the viewer. Basically, I had assumed general compatibility of my scene on different devices. However, that was not been the case. I had figured that it might be easier to show the AR extension on a large iPad and pass it around. Since I had only tested the composition on the laptop and my iPhone, I was very surprised when I realised that several elements of my composition seemed missing on the iPad. For my next AR work, I will use a wide variety of devices for testing in advance. Likewise, there was a problem with the smartphone of the respective user. Adobe Aero seemed to have been tailored mainly to the IOS operating system, as users trying to scan the QR code with another manufacturer's phone had difficulties downloading and opening the content. This issue might also have been caused by the fact that I placed the QR code on a piece of plexiglass to distinguish it more from the wall and the label, but the reflection of the light might have confused the camera.

What was just as surprising was that the download speed of the online scene varied greatly from device to device. Some visitors stood in front of the work for a long time and waited for the download. I'm not sure if it was related to the quality of their internet connection or the available memory on their phone. Anyway, very often it came down to me passing my smartphone to the viewer to enable him to look at the virtual environment. It was also a bit complicating that I didn't explicitly point out that it was an AR experience that could be called up through the QR code.

 

 

 

 

During the installation of all the works, new obstacles opened up. I had stated in the proposal that I would need free floor space, but due to a lack of space this was probably not taken into account for the time being. I communicated with the curators again and showed them my AR composition and they assured me that they would realign the works to take this into account. Ultimately, however, two ground-based sculptures were still kept in close proximity, so I was always careful to keep the viewer's safe distance from them while exploring my work. I also would have preferred to position the painting a bit closer to the ground as the feeling of stepping into the scene gets slightly weakened by it being lifted from it.

What I think works really well is how they regulated the visual temperature in this space by matching my work with its opposing colours to the left and to the right. This clearly enhances each work.

I will take all these aspects into account in my next exhibition and try to minimise such sources of error by placing a pedestal at the right distance in front of the work and placing an iPad with the complete scene on it, which the viewer can pic up and easily immerse himself in the work.

Installation of the work
Video taken by me
Video taken by viewer 1
Video taken by viewer 2
Interaction and future development

 

The reception of my work was consistently positive. The AR component in particular surprised many visitors to the exhibition. They informed me that they had never experienced such a connection and interacted extensively with the entire work. Although not all visitors pressed the record button to record their interaction, I was able to obtain three interesting examples of very different interactions.

 

In the first video I produced the recording. It was a test to see how the final scene behaved in the environment. I stayed where I was and panned the camera to capture the entire scene. My focus was solely on the coherence of the composition.

 

The interaction of viewer 1 was completely different and unexpected for me. As he perceives the AR he immediately tries to dive in. He moves his hand around the frame, trying to catch and hold the scene, and even ends up looking at the camera himself. He becomes part of the composition and changes from passive viewer to active viewer. At the same time, AR is reaching its limits here, as it always reproduces the scene in the same environment, regardless of whether something is in front of the camera lens. Because the AR is triggered here by the image of the painting, it only perceives when this surface disappears, but has no sensitivity to new elements that appear in front of it.

 

In Interaction 2, the viewer has turned the smartphone, he uses it like a conventional camera and slips into the role of a director who is looking for exciting perspectives and points of view in order to dramaturgically capture what he sees. He moves with the objects and is sensitised to their spatial relationships to one another. This is not about becoming a part of the scene, nor about fully grasping and analysing it, rather it is an experiencing of the scene that takes place here while the camera becomes the viewer's sole point of view. Augmented Reality becomes the dominant reality and the lens becomes the tool to sound it out.

I would like to further explore the different ways of interaction or reaction to a mixed reality and how the recording device impacts how we approach our environment.

bottom of page