round corners
Space-Time Correlations Focused in Film Objects and Interactive Video

1 A Step into Art History: Space and Time in the Panel Painting
The close relationship between time and space has been a dominant theme in art for a long time – and still is. The inseparable link of the two dimensions already played a significant role in the religious representations of the Middle Ages. In iconographic paintings omitting a spatial frame of reference, i.e. a spatially worked out background, a link was established to the time dimension of the representation long before the central perspective was used. The divine countenance in its motionlessness and non-spatial reference stripped off the chronological imagination of reality in order to illustrate a kind of existence that could be termed "out of time", "out of this world", "out of the human", and thus godlike. The work of art in the Middle Ages did not have the function of portraying a realistic time-space structure as it concerned itself with the illustration of a spiritual context.

The story of Jesus' life from the annunciation until his resurrection is a central theme of the Medieval picture. The efforts, not only to represent single scenes from the narrative pictorially, but to picture a sequence of actions happening in different places, led to complex programme paintings such as altar paintings which were organised around a combination principle.
The principle applied was not a chronological sequence of pictorial representations, as it is the case in common modern picture stories but the relation of single turning points in the story to each other. Although in some paintings the foreground and background were worked out in perspective, those were no attempts to portray the world in a realistic manner but to, consciously try and represent impossible space and time conditions intended to point to the extramundane character of the situation and the subjects being portrayed.

The art of central perspective, discovered by Brunelleschi towards the end of the Middle Ages and used to a huge extent in Renaissance painting, defined, in contrast to former painting, the art of the mathematical and the rational in painting. The space defined in a painting constructed with the help of central perspective is a space in which each subject is seen within the context of its spatial presence and in relation to other objects (and subjects). It is an organised space, whose ordering principle - the central perspective - was seen as a guarantor of harmony and beauty in the representation. Corresponding to this enlightened spiritual attitude an attempt was made in painting to combine it with the precision and the consistency of mathematics.

The possibility to determine locations for particular subjects and their distance from each other also implies a different presentation of time. The distance between two subjects in a mathematically constructed space also has a time-related meaning, in that a moment in movement is represented: The representation of the phases of movement, the freezing of a moment within a movement, makes the subjects seem more realistic. They were juxtaposed with each other within a contextual frame in a space-time relationship. The first attempts, to represent movement also through space, show that the space was also understood in terms of its time dimension, although the means of expressive representation of the movement in the painting were still missing.

In the epochs successive to the Renaissance, the Baroque and Mannerism, the movement in and through space became an important artistic theme. Again the Baroque time was concerned with the presentation of key moments within a story, with a short time-cut from within a chronological sequence: What is interesting here is that the attempt to represent movement and thereby time in space, was in principle very similar to the methods of the cubists and futurists in the last century: they dynamised the painting space and represented the movement phases in such a pronounced way that one could imagine that they were attempting to portray the body at all possible stages of a movement.

2 The Extension of Space and Time through the Expansion of the Format
Other artistic concepts attempted to reach an extension of space beyond our limited eye view. Panoramas, that were discovered at the end of the 19th century, string together spatiality and break the central perspective through isometry or through vanishing points in the painting. In those days one could document landscape exactly through the use of technical aids such as the pin-hole camera. Naturally the reality of a panorama was limited to the the human's eye view so that in the process of viewing the viewer had to turn on his/her own axis. Thus the illusion of a real landscape was evoked for the viewer. This effect must have been even stronger with the moving panoramas, which, in front of the audience, were slowly hauled from one roll placed on one side of the stage to another one on the other side, for example to simulate the movement of a ship. Thus panoramas can also be seen as the precedents for films and motion rides.

Panoramas represent therefore not only the extension of space through the positioning of stringing space together but also the extension of time through the build-up of chronological time. The simulation of a boat trip down the Mississippi was both a sequence of spaces and actions within a clearly defined time period.
Although moving panoramas are nothing but stills proper, they helped to produce the illusion of movement by causing the viewer to move his/her eye over the picture, getting a notion of time sequence, or by moving the viewer him/herself, or the canvas.

In photography the slit camera enabled the organisation of time on a plane. The camera takes only a vertical line, a minimal section of space over a restricted time-span. The result is the documentation of different time periods on one plane based on a cut through the space.

3 The Exhibition of Time Periods: the Moving Image
The need to present time and time-based periods in painting and sculpture culminated in the works of the futurists and the cubists, who now actually managed to form sequences of, and to abstract, single phases of the movement of the body through space and thereby produced new dynamic bodies. Looking back on these early attempts to present movement and time at the beginning of the last century makes the discovery of film seem an unstoppable and logical cultural step forward. Film was born out of an ever-growing need, based on more than just one moment, rather to document and represent, respectively, a period in time. Within our visual perception, single pictures shown one after another in fast succession became blended to a moving picture, which occupies one clearly defined temporal frame, but also one place, defined through the position of the camera, the moving camera, the camera angle, the aperture setting and the focus.

The first steps towards film were made by Eadweard Muybridge in the 1880s with his technique of "simultaneous photography". Muybridge used a battery of cameras (between 12 and 24) laid out parallel to the horse's path. A Ssring running from each camera across the track was broken by the horse as it trotted or galloped, successively triggering the cameras to shoot. Then Muybridge's sequential exposures were projected at film speed and brought uncannily to life.
The French physician Étienne-Jules Marey found Muybridge's techniques and results problematic for several reasons. Because Muybridge used a different camera to record each new image of his subject, a single point of reference from which to evaluate changes in the subject's position was missing. Instead, Muybridge's batteries of cameras, placed parallel and adjacent to the subject's movement, kept the relation between camera, subject and background consistent in each frame. It was also difficult for Muybridge to accurately and consistently measure the gap of time between each image, thus making it impossible to render a complete representation of the subject's movement.
Marey had, in Muybridge's work, examples of how not to photograph movement. His solutions, developed over the course of twenty years, were helped by the steady advances in photography as a whole, including faster and more stable film stock. His primary strategy was to record multiple images of a subject's movement on the same camera plate, rather than the individual images Muybridge produced. Marey's most aesthetically compelling invention was the chronophotograph: multiple exposures on single glass plates and on strips of film that passed automatically through a camera of his own design. The results were scientifically important, but also inspired artists such Naum Gabo, and most well known, Marcel Duchamp's „Nude Descending a Staircase“.

The advantage of film was that one was able, for example to document time changes in cuts, whereas the first films that were produced were actually limited. Among the themes of the first short films, which could be filmed in one take with a static camera were everyday stories like the driving of a train into a station (Lumière Brothers) or people leaving a factory.

The next essential step leading to the art works of the present, which rather concentrate on the time- space relationships, and the moving image as metaphor and medium all in one, was the digitalisation of pictures which is a fundamental pre-condition for the real time-interaction with the moving image. The viewer does not need to subordinate to a chronological sequence but is rather able to focus, for example, on single cuts of film or single areas of the picture. Digitalisation and interaction enable an individual and complex representation of space and time frame in the virtual world by navigation along the time or space axis.

3.1 From film to film object
Film material is always understood as meaningful in that it is fixed both in time and geography. Owing to the technical means of today, it has become representable as such in the context of its space-time expanse.

As early as in 1994, Edward Elliot, who worked at MIT Media Lab, developed (in a dissertation under Glorianna Davenport) the so-called "Video Streamer," a tool mainly designed to view video films by presenting single pictures of the videos strung together to produce a picture-block. The block is not seen as a static picture but as a temporary, changeable manifestation of a stream of pictures. The block is composed of one half-minute video, where every new single picture on the front block is visible and with each new shot the pictures move backwards, completely disappearing from the block within half a minute. This viewing tool lends itself to the idea to edit video in a manner different from the traditional way, since the outer area of the block is composed of the outer pixels of the single pictures, and therefore is also "readable", in principle in a similar way as the picture from a slit camera.

A related, but essentially more complex concept was presented in the middle of the 90's in “The Invisible Shape of Things Past”, a work by Joachim Sauter and Dirk Lüsebrink, which explored time related to virtual space and navigation through virtual time.

In this work, film sequences are transformed into interactive virtual works. The transformation is based on the camera settings in a film sequence (movement, angle, and focal distance): The single film shots are strung together along the camera path when it is transformed into virtual space. The angle of the shots to the camera path depends on the angle of the real camera, the size of the single shot on the camera focus used. The pixel edges of the single shots form the outer surface of the film object.

The result is a film object that is based on a complex camera movement and virtual information architecture, respectively, and that can be interactively explored. It is the spatial interface for the information, which it contains.

In a second step, a spatial and time based organisational concept for film objects was developed: As each film sequence occurs not only in one place, but also at one time, a virtual representation of the surroundings was constructed, enabling users to navigate through time. One such construction modelled all urban building structures of Berlin since 1900. The film objects were placed in virtual reality at the place and time of the takes in the respective time dimensions.

The documentation of a particular space and time sequence is thus transformed into a perceptible space-time object with an individual aesthetic quality. Independent of their interactive application, these film objects, produced with the help of these transfer tools, have a particular poignancy because of their form that abstracts the literal picture material. This represents an interesting parallel to the sculptural work of the futurists, who used sectional methods to cut through space and time and achieved really expressive results, especially on a formal level.

Tamás Waliczky’s sculptures dating from the mid 90’s are based on a similar principle as Sauter/Lüsebrink’s films. For them, too, the starting point is digital film, and the results of the works are virtual sculptures, which, however, in this case are definitely only obtained from a space-time development of single visual objects/bodies.

Waliczky extracted the movement of a person from digital film by cutting out the silhouette frame by frame. As a result, the single shots/frames are arranged a row forming a virtual sculpture, in which short moments of life are “frozen”. Waliczky refers to these sculptural works as "Time crystals", through which the viewer can move in different perspectives, at different camera angles, and at different speeds.

The work, "Field-Work@Hajama," 2001, by Masaki Fujihata, also starts from the principle that film material has a space dimension which can be transferred into a three-dimensional representation in virtual space.The starting point are digital video-shots that were recorded in urban surroundings in Tokyo together with the exact GPS data. From this a topographic and time-based system of co-ordinates was produced, since each video frame represents both a defined place and a particular moment of the take. Fujihata translated this system of co-ordinates into a virtual three-dimensional space, in which the video takes are displayed in virtual space in positions corresponding to those in which they were recorded and move alongside these three-dimensional GPS traces. Through the use of the interface the viewer is able to follow the video images and their paths and to navigate through the three-dimensional space and thereby experiencing the complexity of the interrelation of space and time. By the use of stereoscopic projection the viewer can perceive the picture three-dimensionally, from which, in addition to this, he gains an individual experience of the hyperreal information space through the interaction with the interface.

3.2 From film to cut through space
Film a visual concept originates from the correlation of time and space and can be understood in terms of a linear ordering principle, which attributes a particular spatial and pictorial configuration to a particular point of time. We see time as a linear dimension with a definite extension. In our perception, these experiences are congruent with the visual impressions of the film takes.

Both the film objects of Sauter/Lüsebrink and the sculptures of Tamás Waliczky visualise the spatial dimension of film as well. A further step which followed the conceptual idea of this type of representation is the cut through the film's space axis, a process that was developed by Martin Reinhardt under the name "tx-transform“ and was first shown publicly in 1998. In 1997, Björn Barnekow had conceived a very similar project bearing the title of "timemirror."
Tx-transform starting material is a traditional, digitally recorded or digitised film that is transformed with the help of software. In its original form, each single picture shows the entire space within a minimal time-span. By contrast, the transformed single film picture only shows a minimal spatial clipping and its complete development during the recording process. Each single picture is a cut through the space axis, a sequence of pictures produces visual effects such as the following ones: Houses start to move, moving trains become shorter with increasing speed. Thus, items in film are not defined as an representation of some concrete existence, but rather as a state in time. The result of tx-transformation can, depending on the mode of perception, appear completely abstract or completely realistic. Theoretically, these spatial clippings can be positioned deliberately, so that new perspectives and effects develop again and again.

Based on a digital slit camera is another project that combines traditional analogue clock with camera feed, creating a clock face that shows the history of the space it is looking at. “Last” by Ross Cooper and Jussi Angesleva transforms the slit pictures into a clock-like display: The clock's hands are narrow slits of the live video feed, and as they rotate, they leave a "time trace" on the clock's face. The hands are arranged as concentric circles, the outermost being seconds, the middle one minutes, and the innermost hours. Thus, the clock face displays the last minute, last hour and last 12 hours as its spatial history. The video feed can be any live video source: A camera mounted on the clock itself looking at what is happening in front of it, remote camera streamed over the internet or TV signal fed directly to the clock. The clock can thus display the local space, remote space or media space respectively. As a clock, the emerging imagery becomes contextualised and makes it meaningful in the space it is being displayed at. As an installation, the system can be used as a living, aesthetic element reacting to the usage of the space. As a still it documents the different types of spaces, similar to a spatial identity card.

3.3 Penetration into space and time
Camille Utterback and Romy Achituv also dedicated themselves to the representation of the correlation between space and time but their project again addresses the time axis. The installation, "Liquid Time" by Camille Utterback and Romy Achituv was based on traditional video-material that is played, in case of no interaction of the viewer, in the normal chronology of recording. An interaction system based on video-tracking enables the viewer to intervene in the chronology, moving backwards in time within a chosen picture area, or within space, respectively, while the rest of the film runs on chronologically. What Utterback describes by the title "Video Cubism" is eventually the splitting of the video image into multiple time zones.

A further concept of non-linearity and coexistence of different time zones in a picture has been developed by Romy Achituv alone and transplanted into the interactive application "BeNowHere." The panorama of a site was documented from one perspective at different times of the day. Within a drifting section of the panorama, the application shows a single moving picture at each stage. Parallel to the camera view, this picture drifts along the space and time axis and leaves a static picture, a trace of time and space, by means of the "last" pixels from the previous moving picture. The viewer can intervene now by using the input medium in order to focus on a section of the image and to activate the moving picture at exactly that point. This means that the viewer focuses not only on a particular space but also on a particular time, namely a point in time before each activated picture in case it is positioned to the left, and after the activated picture if it is visible to the right.

For Achituv this application demonstrates structural possibilities for non-linear cinematic narrative that break away from the reliance on "montage" as the basic semantic unit of cinema. On celluloid, time and space are inevitably fused: every film frame represents a particular space and a particular point in time. Transitions from one space to another, as those from one time to another, can only be affected through a "cut" (or a "fade" - which is no more than a sophisticated cut). The synchronised scenes of „BeNowHere“ suggest the possibility for playing with the viewers' expectations of space and time by affecting transitions between scenes based on user panning alone. This could be achieved by seamlessly integrating scenes. Instead of three different times of day like it is now, the application could be based upon three different scenes shot with parts sharing an identical backdrop (scenery, props, light etc.) The user/viewer then could push the "narrative" forward by transitioning from one scene to another simply through panning the image in different directions.

As we have seen already in the application “Video Streamer” by Elliot an also in “Invisible Shape of Things Past” by Sauter/Lüsebrink some of the experimental work with video imagery results in possible tools for the analysis and editing of moving images. The same applies to a project that was just recently developed at the University of the Arts Berlin. “4DDurée” by Sascha Pohflepp can either be used as a viewing tool or as an creative means for the manipulation and sculpturalization of images.
By selecting one or more areas within a frame it is possible to create a video sculpture in which the single bulges represent other time zones before or after the main frame. The staggering of frames inside the virtual space of an 3D environment illustrates the 4th dimension of the moving image and creates a virtual object inhering an artistic character in itself.

4 Conclusion
A number of contemporary and more recent art projects have transformed film-material into interactive virtual spaces, in order to break through the traditional linear quality of the moving image and the perception of time, at the same time to represent, or to visualise the spatial aspects of time respectively. In the times of resampling, the concentration on an relatively old picture medium and its transformation into a space-time phenomenon open to interactive experience does not seem surprising. The results of these experimental works exploring and shifting the parameters of the linear film are often oddly abstract and quite expressive in their formal composition, and, consciously elude simple legibility.
The development of space-time representations to the present documents the keen interest of artists to portray space-time correlations in their complexity by using methods of representation available at the respective times. Any obvious step towards an innovative way of representation each time implies a radical break with old concepts of visual art. Both the interaction with film and the representation of the space-time characteristics of film in virtual space, which, in the described works, are fathomed by experimental and artistic means, represent a conceptual expansion of the film medium and a break with traditional perception. In a similar way, for example the panorama picture once broke with the traditional understanding of the work of art of the preceding century.
 complete text with references (28.0 KB)