3D subtitling calls for tectonic changes in all aspects of production and post-production of stereographic films and TV programmes. It requires new plots, new shooting approaches, new conventions and new workflows that will profoundly change the industry, says ALEXEY KOZOULYAEV, Founder and CEO of RuFilms, and a leading light on the subject.

Any subtitle changes the structure of the perception of the flow of information that an audiovisual content in any format originates. The paradox about subtitles is that they belong properly neither to the text nor to the image; they occupy a hybrid and intermittent place that is never fully their own. All too often subtitles are perceived by the TV and movie community as a necessary evil that one has to put up with for the sake of global and international releases and exchanges. Interlingual subtitles are always perceived as a supplement to film.

Subtitling conventions and rules - including time and space constraints, the need for consistency with the image, the special use of punctuation and so on - result in a specific type of strongly edited text which departs from established linguistic norms.

In dubbing or re-voicing, the voices of dubbing actors fully replace the voice track of the film, following rules of lip synchronization. In this way, dubbing aspires to reproduce the impression of authenticity of the movie. Subtitles almost never are equal to either the voice track of the film or its dialogue continuity.

Subtitles are interposed between the viewer and the film, allowing the audio stream fully to be heard by the audience. This important formal difference means that subtitles interrupt the effect of transparency and the concomitant perception of naturalness in film. Thus, the common say within the subtitling industry is that "subtitles should be read, but not seen."

A subtitle or a caption should complement the material it is attributed to, but should never be difficult to read or overpowering. Within a regular 2D presentation this means that a subtitler needs to make sure that any text they put up on screen is clear, legible and conveys the message intended, while not occluding the action and remaining visible only long enough to be read.

Turning point

But there is something else, something more basic about 3D that overrides most prior subtitling considerations: 3D movies and broadcasts in their best forms are immersive. The audiences are literally "drawn" inside the course of the action. There is a strong sensory bond between a viewer and the action.

Subtitles are an element that, in their 2D form, is already strongly artificial in the picture. But the viewers of 2D movies and broadcasts can still live with them and incorporate them into their perceptive flows at little expense. 3D subtitles are different.

If executed with little regard to the specific action on the screen, 3D subtitles can disrupt the entire immersive impression, break the sensory bond, put the viewers on the outside of the action, making the show dull, waste enormous budgets of 3D movies and TV programmes by keeping the audiences away from 3D presentations.

There is evidently a new basic set of constraints that influences (a) the translation processes, (b) subtitling and placement conventions, (c) the overall look and esthetic qualities of the subtitles, (d) the timing of the subtitles, and (e) their applicability to different kinds of content.

Our work and preliminary research for a number of our customers have shown that in order to make 3D subtitles a workable notion one has to concentrate on redefining the entire set of working conventions related to subtitling and, in fact, make 3D subtitling a branch of research in audiovisual translation studies that is set apart from 2D subtitling, dubbing or voiceover.

A no man's land

It is clear that 3D subtitles need to be defined differently and the debate on their placement conventions is still ahead. So far, professional stereographers don't like the idea of subtitles of any type being applied over the top of any 3D material, and would prefer in the first instance that they be removed and professional dubbing or voice over tracks be applied.

In the instances that subtitles must be included, it is the preference of these professionals that they should appear in areas of the presentation where no depth cue information exists at all, such as in the black of a letterbox cut.

It is clear that 3D subtitles interfere with the attempts of most stereographers to create a genuine 3D effect. We all know that if a 3D video is shot, displayed, or viewed poorly depth perception of the objects in the scene may conflict with the 2D depth infor-mation that we perceive. These conflicts can cause the viewer to suffer similar symptoms to common motion sickness (fatigue, headache, dizziness, or in the worst case, nausea).

3D either creates a world of its own with its conventions and rules - like in 3D games and animated movies - or steps in as the closest possible replica of the real world in terms of sensory perception. The stereographers have to invent a number of ways to shoot and to build the plots and the entire narrative to avoid negative perceptual and health effects in the audience. The clinical and marketing research shows that once acquired through negative sensory experiences the aversion to 3D presentations is very likely to stay with an individual for quite a time.

The issue then emerges - if they are so alien and disruptive for the 3D presentation, who needs the subtitles in the bright 3D world future? Why can't dubbing and revoicing replace them?

Well, subtitling is cheap. It will be cheaper than dubbing and revoicing as there are constant attempts to automate different aspect of the process that are ever more successful.

I'd say that in the case of 3D: There are domains where subtitles are a good and acceptable option and there are cases in which stereographers aiming to produce a global release should either work with subtitlers to incorporate the subtitles into the texture of a movie or exclude them entirely from possible localisation options.

The question of using, not using, or finding new conventions for 3D subtitling goes far beyond the movie realm. Currently, 3D and 3D subtitles are used in the gaming industry even on a broader scale. After all the gaming market’s turnover worldwide almost exceeds the cinema box office and video sales.

Still the mechanistic transfer of 2D subtitling conventions into the 3D world lead to a serious negative impact on the commer-cial potential of 3D games, a claim confirmed by NVidia. The company reports that many of those 3D games that are judged to be poor for 3D viewing have problems with subtitles/burnt-in titles that kill the stereoscopic impression.

Game developers are caught between the need to localise the games for new markets and the prohibitive costs of dubbing if they want to avoid subtitles (especially in the case of the so-called "sim-ship" global releases).

At the same time this avoidance of subtitles seriously hinders the expansion of 3D movies and games into new markets. Let's remember that it has been the cheaper subtitling and voiceover options that allowed for the boom of satellite TV channels in new national markets and the localization of the games.

Role redefined

Before we proceed to redefining the role of 3D subtitles in the localization processes applicable to 3D audiovisual content, let's take a look at the interaction of 2D subtitles with the cinematic and game dataflow.

The naturalisation of the cinematic codes, through which the effect of the real is produced in film, is carried out by a series of visual sutures. The cinematic code, thus, seems natural and eventually becomes naturalised, that is, it sustains national and cultural audiovisual data presentation traditions.

It is this situation which subtitles challenge. They halt the process of naturalisation and suture by adulterating the image and the editing of the film. The list of tricks used by subtitlers to ensure the unity of image, sound and text is long, and is almost always imposed from above, that is, the subtitling companies. They naturally play down the alienating effect of subtitles.

Are film authentication strategies different for 2D and 3D subtitles? I believe the answer is Yes. Strategies have to change and are indeed changing. That is the issue over which most subtitling companies are currently pondering.

In his article On the Defamiliarizing Effect of Subtitles, Dionysis Kapsaskis says "The interest of interlingual subtitling as a cultural fact lies primarily in its bringing together, literally in the same room, two disciplines - translation and film - and at least two linguistic (and often national) traditions."

Subtitling therefore constitutes a unique forum not only for the comparative linguistic examination of such traditions, but also for an assessment of the representations of these cultural traditions. There is a very important threshold that divides 2D and 3D subtitles in this respect. 3D subtitles are much strongly related to the cultural traditions of image and moving image representation as well as to national film authentication strategies and techniques. Why?

Goal redefined

3D subtitles interact with the film (and game) environment in a much stronger fashion than 2D subtitles and become elements of the action and shots themselves. Their emergence on the screen has to be contemplated from the moment of their arrival into the shot unless producers want to disrupt the stereoscopic impression. Thus, we must assume that 3D subtitling is a strong artistic undertaking. This means that 3D subtitling standards need to incorporate:

- Technical considerations (number of lines, characters and safe zones);
- Artistic considerations (appropriateness of subtitles in certain positions and in certain contents) considerations;
- Linguistic considerations (correct and vivid language of translations).

If we try defining directives for 3D subtitling based on the principles of 2D subtitles, we face considerable challenges. The need to keep near subjects away from the edge of the frame (where the picture for one eye could leave the frame) leads to the "safe zone" becoming much narrower than for 2D subtitles, so the translation must be heavily elliptical.

There is the need to minimise the use of a narrow depth of field. 3D subtitles (when projected in the same way as 2D ones) create an undesirable narrow layer on which the viewers have to focus their attention. The demonstration of unsuited 3D subtitles, thus, has to be as short as possible.

Our own research and that of Screen Subtitling Company in the UK show that viewers watching a visually more demanding 3D movie requires more time to re-focus on the subtitles at a given depth, then back to the action on a different plane. Stereo convergence has been seen to take anything up to about 1-2 seconds each time.

This means that a given subtitle must be present on the screen for perhaps an additional three seconds depending on the difference between the depth of the action and the chosen fixed position of the subtitle. But, this length of additional time is unaccep-table for most presentations.

Converted 2D-to-3D subtitles are reported by viewers as "irritating"?, "distracting"? and in cases of highly intensive action on the screen "annoyingly unnecessary."? As they are not adapted linguistically, none of these 3D subtitles is read. There is not enough time for that and the "converted subtitles" are perceived as very "verbose" although the same subtitle track in a 2D "pre-op" original version evokes none of these hard feelings.

This contradiction between "as short as possible" and "additional three seconds" cannot be resolved by linguistic or technical means. There are artistic solutions to this paradoxical issue of 3D subtitles.

Russian TV and film directors already experimented with subtitles becoming parts of the movie imagery and, in the near future, of the 3D scene. In Russian vampire movie Night Watch by Timur Bekmambetov, subtitles and credits were put on roadside billboards, thus becoming indispensable parts of the scenes. We suggest this approach be called 'Image Blending' of 3D subtitles. But this is not the only option.







If we posit that 3D subtitles are an artistic phenomenon, it means 3D subtitling has deep cultural roots in its perception. If so, in our effort to elaborate country-specific standards of 3D subtitle placement one ought to study the cultural roots of the placement of texts in artistic representations in different cultures.

Indeed, different cultures over the centuries formulated different ways of placing text and imagery. We can single out several distinct representation modes.

- The Western traditions (Medieval/modern, i.e. comic-book like).
- The Chinese/Japanese tradition of blending paintings and hieroglyphs
- The iconic Russian/Orthodox tradition developed in 1920s in the Soviet poster culture
- The Moslem pattern-blending tradition

Far from being the sole preserve of academics, these considerations can be put to practical use. Screen Subtitling is developing a software (3Ditor) built on the concept of "Dynamic Positioning" that relates 3D subtitles closely to the events in the picture. Subtitles start resembling bubbles from comic strips. They move in space in relation to the action and point of interest occurring on the screen. In some cases, the subtitles can even 'follow' the action within a scene. However, these "bubbles" are anchored in a particular culture, which determine their acceptability to the viewers.

It is evident that "Image Blending" and "Dynamic Positioning" of 3D subtitles do not exclude each other. They are complementary processes, but the exact mix is defined by the basic audiovisual product. What unites them is the rejection of "simple" 2D-to-3D conversion of the subtitles by creating a fixed position for them "somewhere in front of the image" with no regard to that image. As stereography develops we're certain to witness other natural and complementary modes of subtitling being developed in respective cultures.

Actually, the practice already shows that there are formats and genres that accommodate 3D subtitles more than others. The number of genres where 3D subtitling can be comfortably used currently is defined by the ability of the source "reality" to be:

- Constructed from scratch - like in animations and 3D games. There, if a subtitle is necessary, but it doesn't fit well, the reality may be reconstructed to incorporate it (including prospective point-of-view etc);

- Adjusted, "bent" and orchestrated to incorporate 3D subtitles - sports events, concerts, mass events. In 2010 and early 2011 at least two 3D sports channels were launched in Europe - in the UK and Finland.

In these areas it seems logical now, while planning a 3D project, to include the localisers (subtitlers) into the planning process from the very beginning - especially in gaming - so that the creative team could make, say, an expert choice between Dynamic Positioning and Image Blending of 3D subtitles in the future product.

The way forward

A stereographic subtitler works with due regard to the translation of speech into writing, but he also translates one representation system of signs and images into another one. Here, the things that matter are not only linguistic barriers, but also differences between representation modes in different cultural environments.

When viewed from the point of view of artistic conversion expertise, 3D subtitlers at large are not any longer seeing themselves cinematographic "also-runs"?, eternal outsiders to the film and game creation processes.

Their contribution is needed from the start to create a non-conflicting and easy-to-adopt 3D environment and to avoid wasting budgets on scenes and episodes that may ruin 3D immersion if subtitled for global audiences.

3D subtitling is another proof that stereo-graphic films and TV programmes will require profound and even tectonic changes in all aspects of production and post-production, requiring new plots, new shooting approaches, new conventions and new workflows, that will profoundly change the industry.

Director Timur Bekmambetov turns film subtitling into an art

Conventional film subtitles hover limply at the bottom of the screen. If you're lucky, they're legible, and hover at the right moment. Mostly, they're not, and they don't, writes reviewer Alice Rawsthom in The New York Times (27 May 2007). [But] there is nothing conventional about the subtitles in Timur Bekmambetov's movie Night Watch.

When Fox Searchlight decided to release the film internationally, rather than dub the original into English, Bekmambetov insisted on subtitling it and took charge of the design process himself. He conceived the subtitles as "another character in the film, another way to tell the story," rather than treating them as extras. The result was sensational.

Most of the subtitles in Night Watch are in clear, white sans serif lettering, but others adopt different guises to reflect the mood and action of the film. Some glide across the screen from different angles. Others glow in red, if a character is especially angry or agitated [...]. One explodes in a puff of smoke after an explosion. Another, uttered when a character heard voices in his head while swimming, dissolves like blood in the water of the pool.

[...] Bekmambetov always ensures that his "special" subtitles enhance, rather than detract, from the movie.



ALEXEY KOZOULYAEV is CEO and co-founder of RuFilms LLC, the leading Russian company in the subtitling domain. The company is active in the fields of translation for subtitling, subtitle format conversion for different digital media and, lately, 2D-to-3D subtitle migration. Last year, RuFilms was the first company in Russia to create a Blu-ray ancd 3D subtitling divisions. The current list of RuFilms clients includes 20th Century Fox CIS, Deluxe Digital Studios, Panasonic in-flight AVOD systems and other major players in the market. Prior to founding RuFilms, Alexey worked for Worldnet TV creating and managing the nationwide TV programme distribution network in 1993-1999 and preparing the large tape library for broadcasting purposes by dubbing, VO and subtitling. In 1991-1993, he worked as a special correspondent for the Russian National TV. Alexey is the author of a number of fiction and non-fiction books and is one of the leading movie and TV interpreters in Russia. Contact. www.rufilm.ru

Story filed 30.08.11