 |
|
|
|
|
|
|
|
|
|
Segmentation and Reassembly of the Digital Moving Image An artistic method of depiction of temporal and spatial nonlinearity The theory of the image and of emblems is a prominent field in temporary cultural theory. For some time now, discourse about the image has been liberated from pure aesthetics and has been adopted by the complementary practice of media science and its phenomenological research. Within this con-text, the image is discussed across disciplines as a fundamental method of world construction. Current research on imaging techniques aims at ascertaining what impact these techniques have on our un-derstanding of the image and on how we use the image. Thereby, for instance, it has been under-stood that the development of central perspective went hand in hand with a new concept of subjectiv-ity and that photography in its early days was dedicated only to the recording of the real. In 1938, Heidegger gave a lecture called “The constitution of a modern view of the world through metaphysics” in which he evaluated contemporary imaging techniques developed by the natural sci-ences as a fundamental change of practice in modern times. Heidegger saw the basis of this change of practice in the visualisation of the world. According to Heidegger, it was only this technological prac-tice that engineered the world as an image and that made the constitution of a world image possible. He clearly differentiated between the “logical image,” produced by the natural sciences through proc-esses of calculation and measurement, and the “syntactical image,” which, as an optical image of the world, is dependent on human subjectivity. The phrase “to be in the picture” demonstrates the character of the world as object by describing the act of understanding as a symbiosis of the subject with its representational image of the world. At the same time, it implicates both the difference between the subject and the image of the world and the subjectivity of a personal view of the world. When speaking of image-based artwork made with digital technology, we often stick to the question of how something was made and forget to ask the more important questions: why was it made and, even more important, what does it say about our image of the world? My hope is that this paper, “Segmentation and Reassembly of the Digital Moving Image,” will serve as a starting point for a discussion of what exactly the cultural and philosophical bases of these artworks are. In my paper “Space-Time Correlations Focused in Film Objects and Interactive Video” (2002), I tried to link contemporary artistic experimentation in the field of the interactive moving image to the subject of time and space representation in art history. In it, I showed that the subject of space-time relationships has always played a dominant role in art and presented some major artistic methods of its models of representation, for instance in panoramas implying an extension of format, and in interactive installa-tions. With the invention of the moving image, which included the concept of space-time correlation, a whole new field of artistic experimentation emerged, using film as a basis for the transfer of space-time corre-lations into audio-visually perceptible representations. In this field of artistic research, the digital video image is connected to generative processes and com-putational design, thus closing the gap between opposing genres. In 1993, Toshio Iwai created the first interactive artwork using real-time video technology to alter the spatiotemporal reference of the image by scanning each pixel row of a picture and reassembling stacks of rows in the visual result. A computer programme executed various visual manipulations, such as time-lapse delay, slow motion, and time compression, which were displayed on a number of monitors. This early work laid the foundation for a series of artworks that made use of the segmentation of the digital image into pixel rows, thus creating distortions of the original image and nonlinear spatiotempo-ral representations of the world. Iwai’s main interest seemed to lie in the power of the visual effects , and in fact this newfound technique provided a sense of playful immersion and expressiveness. Nev-ertheless, the application of such a technique in an art context is not only a function of its availability. If we assume that every invention and application of a cultural technique is closely linked to a specific change of cultural precepts and needs--as has been assumed in the case of central perspective--there must also be an underlying cultural concept behind this particular technique. I would put forth a number of precepts that seem to me to lead to this playful experimentation with the digital image and the manipulation of its spatiotemporal dimensions. The first and most obvious is the basic change in the perception of time and space in the modern era. Today’s expanding global net-works, enabling the real-time exchange of emotions and thoughts over huge distances, fundamentally influence our understanding of time and space. They have led to an unconscious adoption of their characteristic speed and have fostered the sense of shrinking geographical distances. The awareness of simultaneous existences and of multiplicity of views of the world supports the imagination of a paral-lel and fragmented reality that can hardly be understood and perceived as an entity. The fragmenta-tion of the world goes hand in hand with the disaggregation of linearity. The impossibility of represent-ing and reflecting the word in all its multiplicity encourages both a reduction of the individual and self-generated view of the world and, at the other extreme, an intense (self-) reflection of the here and now. Pieces like “Soft Cinema” by Lev Manovich/Andreas Kratky and “Somebody, Somewhere, Some Time” by Maurice Benayoun seem to be born out of the same idea, though they use the more traditional technique of collage, which already has a lively history in (video) art. “Soft Cinema” displays various streams of moving images on one screen, arranged along a clear and geometric grid. This continuous image delivery is driven by specific software that generates the output out of a movie database. “Somebody, Somewhere, Some Time,” on the other hand, assembles still images representing diverse locations, time zones and actions linked through a quasi-narrative. In the latter piece, the image is divided into sections accessible to the viewer, who can “dive into” the picture and its manifold details. Benayoun’s interactive piece betrays deep roots in art history, particu-larly in panoramic paintings. Despite the obvious aesthetic and functional difference between the Benayoun and Manovich/Kratky pieces, in both cases complete, autonomous and representational images are combined to form an associatively linked, synchronous universe of images. In these works, the fragmentation of the world and of a worldview takes place on a metaphorical level, by a facet-like assembly of narratives, which, taken together, represent a universal totality. In contrast, Iwai and his successors don’t refer directly to the multiplicity of the world in its entirety; rather, they reflect on the multiplicity of a smaller unit, the here and now, by assembling segments of one real-time image. Another experimental art project that descends from “Another Time, Another Space” is the interactive installation “Zerrfalten” (Defolding) by Nelson Vergara and Stephan Schulz. This piece questions the depiction of time and space in linear video and photography. “Zerrfalten” contrasts these recording methods with a system that combines compressed moments of time into one large image. The system consists of two video cameras that feed the projection. The projected image is built out of segments of the recorded image of the visitor--out of pixel rows displayed in relation to the movement of the visitor and his/her position in space. The greater his/her distance to the camera, the thinner and more numerous the vertical pixel rows from the recorded image. The closer he/she comes to the cam-era, the bigger the segments of the image. The programme grades these segments from left to right and freezes them, unless they are “overwritten” by another sequence of segments or deconstructed by someone walking very close to the projection. This movement opens the small lines like a curtain to their full frame size. Vergara and Schulz use pixel rows as a metaphor for time compression, for the depiction of “now.” Scientific research has shown that the human perception of “now” lasts for approximately 0.3 seconds. This is the time span it takes the various centres and sections of the brain to find a synchronous rhythm that we experience as the present. Paul Virilio coined the term “intensive time,” which de-scribes an extreme acceleration: the time beyond experience, the technically driven time that lies be-low the 0.3—second threshold and eliminates space. In “Zerrfalten,” a row of pixels as a time unit represents machine time as opposed to human time. Only by adding single moments of machine time in the form of pixel rows, is an image created that the hu-man brain can interpret. Additionally, Vergara/Schulz integrated a fitting model of interaction that ex-pressed the temporal concurrence and overlap of events. Moreover, the single pixel row, as well as the frame it originates from, work as a precise spatiotempo-ral description of the visitor’s movement. In the case of “Zerrfalten,” visual experimentation led to an artistic result that demonstrated the nonlinear quality of space inherent in this method of segmentation and reassembly of the digital image. “Barrington Street” , also by Schulz, uses video footage shot from a moving shopping cart that was pushed down Barrington Street in Halifax, Canada. If there is no manipulation by the user, the pro-gramme automatically selects the middle and vertical pixel row of each video frame and freezes it on the right side of the former pixel row. If the user intervenes and moves the cursor, the pixel row parallel to the cursor is taken out. Thus a change of perspective is implied, providing unique nonlinear views of the street architecture. The nonlinear perspective conveys the impression of space that is dependent both on time and on the movement of the viewer. That time and space were rediscovered in the 1990s as topics in philosophy, physics and art can be interpreted either as a late consequence of Einstein’s theory of relativity finally reaching mass con-sciousness or as a side effect of globalisation, which comprises the ideas of distributed space, multi-tudes, and simultaneity. However, the laws of nature are far less incontestable than we might like. For lack of a general and logical comprehension of time and space, we usually favour a naive and illogical understanding of these concepts. Admittedly, art is limited by its metaphorical language, and its appeal to our senses can only convey a rather simple idea of time and space compared to the true complexity of the subject. Nevertheless, artists feel compelled to find appropriate visualisations and systems that provide a notion of the implications of time theory. Tania Ruiz Gutierrez researched models of time representations and came up with a piece about the question of cyclic time and space. She extracted pedestrians from video footage and looped their movements in an endless and desynchronized proc-ess, while she imprisoned them in a torus-like model space. She programmed the trajectories of these figures so that they will walk forever without reaching the boundary of the model space, since the space wraps back on itself. The pedestrians walk in a straight line and at a constant speed, en-closed in the system and in the frame. The film is programmed to last until the moment when all the possible encounters between figures have taken place. While the Ruiz piece can be described as a film that runs by itself in a predefined way, David Rokeby’s piece “Sorting Daemon” extracts human faces from their real-time contexts and reassembles them in a vibrant collage. A CCTV system connected to a computer system recognizes moving people on the street; when it finds what it thinks might be a person, it removes the person’s image from the back-ground. Then, the extracted image of the face is divided up according to areas of similar colour. The resulting swatches of colour are next grouped according to hue. The original faces first appear at the bottom of the collage and slowly separate into the coloured regions with their appropriate locations. According to Rokeby, the focus of this piece lies both in exploring the generative process and in re-flecting critically on surveillance systems. Nevertheless, it is closely related to previous pieces by Rokeby, such as “Seen” and “Taken,” both of which document and visualise the spatiotemporal occur-rences of some site in real time, employing feedbacks, delays, processions of motion studies and loops as well as the extraction of persons from the source image. The fragmented images of “Sorting Daemon” therefore explore not only the metaphorical decomposition of a person by the recognition and classification of phenotype through software, but also embody the fragmentation of existence and of the world in general. The aesthetics of visual and sonic noise is well-established in today’s art culture, but it has recently been given new life by the development of real-time video-processing techniques and equivalent sound tools. In his essay for the Dokumenta 11 catalogue, Sarat Maharaj points out that sonic noise is neither the junk of the music system nor the simple antithesis of musical composition, but rather raw data for digital sound manufacturing done without a script in a tense correlation to the visual. Rokeby’s “Sorting Daemon” is a production apparatus for visual noise, for a structured horror vacui, in which the endless stream of fragments overlap, are duplicated, and appear and disappear without any recognizable goal. Comparable to what Maharaj asserts for sonic noise, here the visual composition executed by the programme is neither a strict negation of traditional pictorial composition nor a visual scrap yard of digital fragments. Instead the visual output is part of the predefined generative process, which is what Rokeby is primarily interested in. This ideological shift--from product to process--is an inherent differentia of digital culture and charac-terises also the following artworks, in which the degree of segmentation and abstraction of the digital image is further increased. Osman Kahn and Daniel Sauter have employed the segmentation of the digital image into pixel rows in several works, starting with a piece called “We interrupt your regularly scheduled program...” which reinterprets the TV broadcast stream by abstraction and time lapse. They chose television as an image source because of its huge impact on society. By switching through daily TV programmes the viewer collects a mass of fragments of information which he/she has to process somehow. It is well known that the mass of information that the human brain must process has never been bigger than it is today. This continuous, nonlinear intake of information has significantly influenced our perceptions of time and chronology and has fostered the development of spatial information management systems. In the piece, a stream of lines is created by first generating a pixel average of every horizontal pixel row for every frame. Then a one-pixel-thick slice of every one of these horizontally averaged pixel rows is kept in memory and scrolled down the projection. The result is a perpetual flow of abstract row patterns that pulsate and change colour and speed in relation to the source material. In his provocative though dated essay “The work of art in the age of mechanical reproduction,” Walter Benjamin compares film with painting and finds film wanting: “The painting invites the spectator to contemplation; before it the spectator can abandon himself to his associations. Before the movie frame he cannot do so. No sooner have his eyes grasped a scene than it is already changed. It cannot be arrested.” Of course, Benjamin was referring to contemporary narrative film; however, today’s experimental art-work with the digital moving image goes far beyond figurativeness and narration, and sometimes it even links itself back to abstract painting. André Bazin was one of the most influential proponents of the idea that cinema, under the aegis of photography, “freed Western painting, once and for all, from its obsession with realism and allowed it to recover its aesthetic autonomy.” Following this train of thought, video liberated cinema from the bondage of narration, and now and finally, the digital moving image frees itself from linearity and fig-urativeness. Through both the abandonment of narrative and the reintroduction of complete abstrac-tion, digital art regains its contemplative and auratic nature. One last example of the segmentation and reassembly of the digital image is a project by Jeremy Welsh that intends to bridge the fields of painting and digital media. One of the results of this project was the exhibition “Dialogue/Transition,” which was mounted in collaboration with the painter Jon Arne Mogstad and the media artist Trond Lossius, whose main focus is on sound. The database behind the project contains numerous photographic images of container terminals that represent temporary architecture and urban zones and thus function as metaphors for transition and transience. By applying several effect filters and video editing programmes, these still images are heavily manipulated: For instance, a single row of pixels is extracted and stretched across the entire image area and then imported into a video editing programme where they are treated as frames of a single-cell animation. Also, more complex geometric forms are achieved by combining several streams of animation in a single video image consisting of multiple layers. The visual results of this image processing are strong rhythmic and repeating patterns which are read and then translated, through real-time sound processing, into ambient sound. The visual output of this process takes on the quality of “painting over time,” comparable to the result that Kahn/Sauter achieve through the live processing of digital moving images. The abstract digital moving image thereby gains the contemplative quality whose absence Benjamin complained of. The segmentation and reassembly of the digital image, the extraction of pixel rows and the “compres-sion” of the image, is used in various ways mainly to express the temporal quality of the world. This method produces a broad spectrum of visual results, ranging from figurativeness and readability to pure abstraction and contemplation. The experimentation in this field of artistic production is fostered both by the artists’ search for new means of visual expression beyond the narrative and their explora-tion of the visual capabilities of the video image and its tools. Within this artistic field, the focus of the work often lies in the live process of translation and transformation from one level of perception to another. Moreover, the work is based on a serious examination of complex space and time correla-tions whose perception and understanding has changed deeply through the everyday use of locative media. While in mass media reality is heavily staged and human perception is exposed to a constant over-flow, the artworks I have discussed here reduce the figurative digital image to its technological basis. The segment of the digital image so treated is then fed into a generative or interactive process, thus building a link between the previously opposing genres of the moving image and real generativity and interactivity. By applying one basic technical method, i.e. the extraction of pixel rows, and combining those “com-prised” moments of time in one moving image, a variety of artistic results is achieved. While the visual products of this artistic method strongly relate to each other and to contemporary painting and photog-raphy, for instance to present-day pinhole camera photography, the cultural basis for this kind of work is the acceleration of daily life and the disbandment of distances. In an even more technology-based future, the digital moving image that today plays a dominant role in culture already will be liberated from darkened rooms and monitors and the rectangular shape – thus leading to new artistic models representing the complex concepts of time and space. |
||||||||
|
||||||||
 complete text with references (56.5 KB)
| ||||||||
|
||||||||
