When a Good Girl Goes to War: Claire Adams Mackinnon and Her Service During World War I – Heather L. Robinson

Abstract:

Claire Adams Mackinnon and her contributions to the war effort 100 years ago are largely forgotten. The product of two Canadian military families, she put aside her burgeoning film career when war broke out to train and work as a nurse before returning to the silent screen. This article examines available evidence to reconstruct this period of her career (1914-1919), encompassing her nursing experience, her role in a fundraising drive for the US Red Cross, and her starring role in a government sexual hygiene campaign, which ignited one of the first censorship storms of the early film industry. I will argue that the choices Adams made reflect not only a determination to aid the war effort, but also place her at the vanguard of the women’s movement during the volatile years of the mid-1910s. This successful actress, who spent the second half of her life in Melbourne and regional Victoria, has largely been forgotten by film history. However, by placing this early period of her work within a firm historical context, one dominated by the fight for women’s suffrage, sex education and the First World War, we gain an appreciation of her significance within the early film industry and the origins of her ongoing community service.

 

Introduction:

For anyone with a passing interest in silent film history, Claire Adams Mackinnon is an intriguing transnational figure. A British subject for much of her life, she was born in Canada to an English father in 1896. A singer and an actress, on both stage and screen, she was a child performer using her real name “Beryl” Adams before gaining success as a motion picture actress in New York. As a teenager in early Thomas Edison productions she was known first as “Clara” then “Peggy” Adams before publically assuming her family pet name “Claire” in 1918. In 1920 Adams moved to Los Angeles and enjoyed an eight-year Hollywood career. In 1931 she became an American citizen prior to spending the last forty years of her life as an Australian.[1] Today, however, she remains largely unknown, despite appearing in some of the silent era’s most influential, popular and profitable films alongside some of the best-known artists and producers of the era.

If Adams is remembered at all it is within select circles in Victoria, Australia. Following the death of her first husband, Adams enjoyed a whirlwind courtship and marriage to Donald Scobie Mackinnon, the scion of a Melbourne legal and horseracing family. Together they reinvented the Western District homestead Mooramong, creating a jazz age delight redolent of the Hollywood Hills lifestyle she had left behind. Now in the hands of the National Trust of Australia (Victoria), Mooramong remains defiantly anachronistic in a district better known for its bluestone piles and weatherboard restraint, a monument to a much-loved couple reflecting their sophisticated tastes and cosmopolitan life experiences. Mrs Mackinnon’s career is reduced to a prelude to her second marriage, like a folly of youth indulged before her adult life began.

Figure 1: Mooramong homestead, Skipton, Victoria. Image used courtesy of National Trust of Australia (Victoria).

Figure 1: Mooramong homestead, Skipton, Victoria. Image used courtesy of National Trust of Australia (Victoria).

An intensely private person throughout her life, Adams was enigmatic from her earliest interviews and has left little evidence to enable a simple or personal interpretation of her career.[2] Unlike many of her more famous contemporaries, she was not a movie star created by a studio publicity department: she was an experienced working actress, appearing across genres for a number of companies and eschewing the celebrity lifestyle. This independent approach resulted in a comparatively low public profile.[3] Nor was Adams in Hollywood when the studios began to construct their historical legacies. As such, she appears infrequently as a footnote in popular and scholarly accounts of the silent era, including the one written by her first husband, Benjamin B Hampton.[4] Eminent film scholars welcomed Hampton’s text as an authoritative account of the history of the early film industry.[5] If even her husband did not think she warranted more than an image caption, one could easily conclude that Adams was not terribly significant. However, as Lesley Speed suggests, Adams’s career is significant not only for the continuity of her workload, but for the time in which she was active, “spanning a period of major changes in the American film industry, including the transition from short films to feature-length productions, the industry’s relocation from the East coast to Los Angeles, the establishment of the star system and the formation of vertically-integrated major studios” (Speed 2015, 4).

Like the majority of women who dominated the ranks of the early motion picture industry, Adams has fallen through the gaps of film history’s indeterminacy.[6] However, as more scholars analyse the period and primary sources become more accessible, we are able to reconstruct a more accurate representation of the industry, the individuals and the era during which Adams was active, and the picture that emerges is one dominated by the women’s movement. Women made up at least 83% of the cinema audience at a time “when women’s voices were particularly valued” (Stamp 2012, 6). Women’s history scholars note that from the late 1910s “young women could not help but be influenced by the currents of the age” (Banner 1974, 151). As a young woman in New York City during the First World War, Adams was surrounded by the opportunities, debates and challenges confronting the first generation of young women free to pursue a career as a choice rather than a necessity. She was amongst the target audience for orators and activists such as Margaret Sanger promoting the birth control movement and key figures campaigning for Women’s Suffrage, which would not pass into legislation until 1920. We have no evidence that Claire Adams identified herself as an early feminist. However, through her career choices, she reflects the values and ideologies synonymous with the movement, a cultural catalyst which grew in parallel with the development of the motion picture industry.

By mainly focussing on two of her early feature films, this article will demonstrate that Claire Adams deserves recognition not only as a well-respected, adventurous and greatly admired actress of her day, but also as a young woman navigating the perils and opportunities opening up for women in the early twentieth century. She is also a representative of the forgotten contributions of countless women working in early cinema. I will examine her work of one hundred years ago, when she stepped away from Edison Studios and the early motion picture industry to train as a nurse, only to return to motion pictures in support of the troops heading off to the battlefields of Europe. By placing these early works within a firm historical context we will gain a more complete understanding of the scale and intersections of her careers, as both nurse and actress, her role at the vanguard of contentious social issues of the mid 1910s and her active participation in landmark moments of the early American film industry.

Real Life Drama

According to the Australian Dictionary of Biography, Claire Adams worked as a nurse for the Red Cross during World War I (Maxwell, 2000). This description conjures evocative images of mud and blood, battlefields and field hospitals. What drove an aspiring young actress away from the movie studio and into a hospital theatre? What data is available to determine how much official nursing she contributed to the war effort one hundred years ago?

Figure 2: Figure 2. A young ‘Clara’ Adams (centre) with co-stars from left: Marion Weeks, Yale Boss, Elsie McLeod and Gertrude McCoy in The Office Boy’s Birthday (Charles M. Seay, 1913). From Kinetogram, 1 January 1913. Source: Seaver Centre for Western History Research, Natural History Museum of Los Angeles County.

Figure 2:  A young ‘Clara’ Adams (centre) with co-stars from left: Marion Weeks, Yale Boss, Elsie McLeod and Gertrude McCoy in The Office Boy’s Birthday (Charles M. Seay, 1913). From Kinetogram, 1 January 1913. Source: Seaver Centre for Western History Research, Natural History Museum of Los Angeles County.

In 1912, Adams began performing for Thomas Edison’s film production company under the stage name “Clara” Adams and appeared in 17 motion picture shorts in two years (Internet Movie Database, 2015). At the outbreak of war in 1914, she was almost 18 years of age. Both her grandfathers had served in the British military in Canada and India before entering business and public service (Letourneau 1982; Manitoba Historical Society 2009). This family tradition and tales of exploits on the battlefield inspired Adams and her brother Gerald as children (Will 1978, 30). As young adults, they were determined to emulate their forebears and sought opportunities to volunteer for the war effort.[7] Together they had grown up in an environment that was conservative and under the influence Anglo-Victorian social strictures of duty and community service. These strictures included a strong sense of volunteerism, which saw upper- and middle-class young people from across the British Empire come forward to support their King and Country (Quiney 1998, 190). Adams set aside acting to make an active contribution to the war effort.

Like thousands of young women across the British Empire, Adams applied to join the Voluntary Aid Detachment or V.A.D service, a corps of nurses and ambulance drivers trained and managed for the Red Cross by St John’s Ambulance. However, owing to her youth and inexperience, she was turned away (“Claire Adams, Starring in ‘The End of the Road’ Is Real Western Canada Girl”, 1920). She could not change her age but was able to address her skillset. Adams enrolled at Detroit’s Grace Hospital Training School for Nurses, one of the most respected institutions of its kind in North America (“Claire Adams, Starring in ‘The End of the Road’ Is Real Western Canada Girl”, 1920). Grace Hospital was just over the border from Canada on the shores of Lake St Clair, close to Toronto where she was living with her father Stanley Adams (Sweet & Stephens transcript). By 12 June 1915, Adams was listed on a ferry passenger manifest as an eighteen-year-old student nurse travelling between the two cities (Ancestry.com, 2009).

Unfortunately the enrolment records for the Grace Hospital Training School for Nurses from this time period do not survive, but the institution’s prospectus does, specifying a strict and rigorous lifestyle for their live-in student nurses.[8] This training would equip the young ladies to assist senior staff with both male and female patients in the hospital’s surgery, children’s ward and the obstetrics wing. Other extant records from Grace Hospital include the lists of graduates, and Adams’s name does not appear there either.[9] However this would not have prevented her from gaining employment as a nursing assistant, especially at a time when people with any medical or first aid experience were in high demand. Many students enrolled and failed to make it through the rigors of training or were unsuited to the prohibitive lifestyle required of those living in (Kathleen E. Schmeling, pers. comm.).

The Canadian Red Cross website describes how anyone working as a volunteer at a hospital during the Great War was referred to as a Red Cross nurse. Unfortunately records of the thousands of people who gave their time and energies to help have been either destroyed or did not exist in the first place.[10] What do still exist though are the Canadian census records taken in 1916 (Ancestry.com, 2009). They record that Adams was staying with her mother in Winnipeg and working as a “doctor’s assistant”. As happened across the Empire, she was most likely one of the thousands of women who volunteered to work in their local hospital. Deer Lodge was one of many country houses and hotels across Canada converted to military hospitals dedicated to convalescent soldiers returning from the front (Deer Lodge Centre, 20014). The English Government was short of both funds and infrastructure to deal with the number of wounded generated by the new means of mechanised warfare. Deer Lodge was established as a repatriation hospital in Winnipeg in October 1915 following a request made by the English Government to Commonwealth nations contributing troops to the war. Each country or dominion of the British Empire became responsible for the convalescent care of their own when casualty numbers exceeded British expectations and resources.[11]

Moving back to Winnipeg to reside with her mother Lillian, a piano teacher, also offered Adams the chance to participate in another historical event. In January of 1916, the women of Manitoba Province were the first in Canada to be granted the right to vote in municipal elections and hold public office. This occurred 2 years before most of their countrywomen and 4 years ahead of women in the United States (Jackal & Millette, 2015). Even during times of war, this would have been something worth celebrating and hard to ignore.

Following the Battle of the Somme in July 1916, the number of wounded Canadian troops requiring convalescent care rose from 2,620 in July 1916 to 11,981 by the year’s end (Veterans Affairs Canada, 2015). Adams may not have experienced the action on the front, but she would have witnessed the consequences it had on soldiers. In whatever capacity she was serving, she would have been expected to meet the demands of a near five-fold increase in the number of shattered patients, some approximately her own age, in the makeshift over-crowded wards ill-equipped to deal with them.

In 1916, Adams was still only 19-years-old. The physical toll of nursing accompanied by emotional exhaustion were the reasons Adams later gave to explain her return to motion pictures, admitting that she “collapsed finally under the strain of her work” (Goldbeck 1920, 27). Although equipped with some training and experience, she may not have been capable of maintaining an emotional distance or remain objective from the suffering she confronted. Well before the Armistice was declared in November 1918, she was back with Edison Company performing for the cameras. In doing so, she continued contributing to the war effort, but it was on a scale that exceeded anything she could have achieved had she remained a doctor’s assistant in Winnipeg.

Returning to the screen in 1917 with the stage name “Peggy”, Adams appeared in 8 Edison Productions. These included Barnaby Lee (Edward H. Griffith, 1917), her first feature film running over four reels (48 minutes), and the first of several in which she was directed by Edward H Griffith.[12] She starred in Your Obedient Servant (Edward H. Griffith, 1917), the first adaptation of Anne Sewell’s classic novel, Black Beauty (see Figure 3). Her next production, Scouting for Washington (Edward H. Griffith, 1917) was a melodrama set during the American Revolution. Wild Arnica and Shut Out in the Ninth (both Edward H Griffith, 1917) show a very young Adams, all dimples and dark curls, revelling in simplistic comedy.[13]

 

Figure 3: Claire Adams with Pat O’Malley in Your Obedient Servant, (Edward H. Griffith, 1917). Edison Conquest Pictures. 1917. Source: UC Irvine Special Collections and Archives.

Figure 3: Claire Adams with Pat O’Malley in Your Obedient Servant, (Edward H. Griffith, 1917). Edison Conquest Pictures. 1917. Source: UC Irvine Special Collections and Archives.

The following year, Adams would make the pivotal films of her early career. They started with a short vignette, one of a series of social satires titled Girls You Know. It would be the last time she appeared in a motion picture produced by the Edison Company. Each sketch “featured a different type of American girl personally selected” by James Montgomery Flagg, the prodigious illustrator, cartoonist and photographer (“Edison Releases Flagg Social Satire Series.” 1917). The First World War was something of a career peak for Flagg. He’d had film experience, was a widely read social satirist and artist and produced recruitment and patriotic propaganda posters for the US government. Flagg’s most recognised image was his creation of Uncle Sam, the grizzled old patriarch on the recruitment posters who “Needs You for US Army”. [14]

Flagg was commissioned to provide the “one-reel photo-sketches … depict(ing) the grace, charm, foibles and frailties of ‘Girls You Know’, the likable types” (“Recollections of ‘Girls You Know’”, 1918). These “likeable types” featured such stereotypical characters as “The Art Bug”, the “Spoiled Child”, the “Bride” and “The Artist’s Model”. Adams was cast as The Man-eater (Jack Eaton, 1918). The influence of the war in this film is obvious. Adams’s character Nina bids a tearful farewell to her uniformed fiancé and dreams of medals pinned to his broad manly chest. Nina is a sprightly little flirt travelling to a picnic with friends, harmless but annoying in her need for male attention. She is eventually “doused off”, pushed into the river, pulled ashore then scolded like a half-drowned kitten.[15]

The Girls You Know vignettes were produced as light and satirical entertainment for the troops and for those on the home front. They also reflect the restrictive caricatures of young women in the early motion picture industry, characterisations of women indulgently pursuing or experimenting with a career (“The Artist’s Model” and “The Art Bug”). Others were defined by their relationships with parents or male partners (“The Spoiled Child”, “The Bride” and “The Man-eater”). As “the Man-eater”, shunned and dunked for her sexually assertive manner, Adams’s character suffers the entrenched social consequences as punishment for her morally dubious behaviour. The cast of the series were promoted as having been “all selected from (Flagg’s) models – the most famous in New York”, though featuring leading women “with no experience but clever ideas and no actresses of note” (“Recollections of ‘Girls You Know’”, 1918). Adams, however, was by this stage a well-established performer, and three of her four “Girls” colleagues also had previous motion picture experience.[16] Flagg, the producers and indeed audiences, may have been more attracted to a fresh pretty face than a talented individual with an identity, reflecting the low opinion many still held of the acting profession.[17] There is no small irony that she was at this time living against tradition, earning a respectable wage and supporting herself independently as a professional young woman.[18]

The Spirit of The Red Cross, (Jack Eaton, 1918)

When the United States entered the European conflict in 1917, the US chapter of the International Red Cross was charged by the government to raise funds and volunteers to maintain an active presence on the battlefields of Europe. The subsequent rise of American humanitarianism and the specific formation of a nation-wide Red Cross campaign coincided with a period when “sensationalistic mass media began to dominate American culture”. These forces, dominated by new motion picture technologies, were responsible for the “reshaping American ways of seeing, feeling, and responding to suffering by treating violence and pain as pleasure-producing commodities” (Rozario 2003, 426).

The Red Cross recognised the timeliness of motion pictures as an effective and expedient medium for stimulating potential donors and volunteers across the country (Rozario 2003, 429). The National Association of the Motion Picture Industry (NAMPI), chaired by famed producer Jesse Lasky, was appointed to the American Red Cross to promote and “interpret the organization for the American People” through the production of a suitable motion picture (“Spirit of the Red Cross 1918). Flagg was engaged to write the script. Jack Eaton, also of the Girls You Know series, was brought in to direct. Adams was cast as the lead, a nurse named Ethel, the image and embodiment of the International Red Cross. Lasky’s production expertise ensured that the resulting film, The Spirit of the Red Cross (Jack Eaton, 1918) presented “some of the best work done in motion picture making” (“Spirit of the Red Cross 1918). With such an accomplished and professional team involved, this 2-reel film was certain to not only hit but exceed the patriotic, inspirational and philanthropic targets for which all parties were aiming.

Adams’s lead character Ethel was the sweetheart of “an American youth” Sammy, played by Ray McKee. When Sammy enlists with the army and sails for France, Ethel volunteers as a nurse. The film portrays the work of the Red Cross in the field, assisting refugees, nursing and caring for the wounded. The New York Times described Sammy heading for the battlefield, envisioning “Ethel in her white uniform, watching over him”, until:

After a charge, he lies on the ground with a bullet in his chest, half conscious. The vision of Ethel awakens him, just as a German comes forward slaying the wounded. Sammy grips his revolver and shoots the enemy. Later, removed to base hospital, Ethel finds him and nurses him back to health. [19]

The Red Cross were not averse to engaging exaggerated propagandist images for the benefit of their cause.[20] Their tactics owed much to the pulp media, sensationalist newspapers and war magazines, aiming to provoke the outraged and patriotic public into “acts of benevolence” (Rozario 2003, 430). What makes this campaign remarkable is that it was the first time such techniques had been mobilised so effectively on a broad national scale, utilising the emotive mass cultural medium of motion pictures.

The American Red Cross expected their national fund raising drive to receive “considerable impetus” from the release of The Spirit of the Red Cross (“Spirit of the Red Cross”, 1918). Newspaper listings from across the country noted that cinema proprietors were donating all takings from the screening to the cause.[21] The New York Times quoted that the goal of the organisation was to raise one hundred million dollars, or around $1.8 billion in US dollars today (Measuring Worth, 2014). By the time the Armistice was declared in November 1918, only 7 months after the film’s release, the Red Cross had raised approximately four times that sum (American Red Cross, 2014). The film is credited with playing a major role in the campaign’s success. By the war’s end, one-third of the US population, approximately 39 million people, were either contributing members of the Red Cross or serving as volunteers.[22] At the cessation of hostilities, war movies were the most successful types of films in the country and “spectacle was in the ascendant” (Rozario 2003, 439).

The promotional poster Flagg created for The Spirit of the Red Cross depicts the diaphanous vision of Adams’s character floating over the battlefield. Ethel valiantly directs stretcher-bearers toward the unseen wounded. The plaintive caption declares: “Not one shall be left behind” (See fig. 4). For the many, perhaps the millions, who attended the film or saw the poster, Adams represented the thousands of nurses serving in the field, saving the fallen and tending the refugees, even though she was not entirely happy with her performance (Lake 1922, 51). She may not have been directly ministering to the wounded any longer. However, through her contribution to this film, Adams achieved more for the war effort as a nurse on screen than would have been possible had she remained one in life. Well into her old age it was commonly accepted and reiterated in biographical entries that Adams had been a nurse for the Red Cross during the First World War. Having taken part in such a high profile and hugely successful campaign, it is fair to say that Claire Adams was not simply “a nurse” for the Red Cross: she was The Nurse, the face on the movie poster, the selfless guardian of the wounded and indeed the embodiment of The Spirit of the Red Cross.

Figure 5: Figure 4: The Spirit of the Red Cross (James Montgomery Flagg, 1918). Source: World War 1 Posters from the Elizabeth Ball Collection, Ball State University Archives and Special Collections. Ball State University, 2011. All rights reserved.

Figure 4: The Spirit of the Red Cross (James Montgomery Flagg, 1918). Source: World War 1 Posters from the Elizabeth Ball Collection, Ball State University Archives and Special Collections. Ball State University, 2011. All rights reserved.

Adams was well positioned to maximise the opportunities offered by both the emancipation of women in general and the freedoms and opportunities offered by the motion picture industry. At the dawn of the Twentieth Century, middle-class women in North America were still expected to abide by the traditional Victorian roles of marriage, motherhood and marital obedience. As the 1920s loomed, however, “a woman’s alternatives to marriage were not only possible but exciting” (Banner 1974, 48). This social shift was influenced by women’s experiences backfilling male positions in the workforce and volunteering for the war effort (Quiney 1998, 193). By the time Armistice was declared, Adams was twenty-two years old with an independent motion picture career, an exciting new option embraced by increasing numbers of young women in the West. As such she exemplifies the symbiotic relationship between two new cultural forces at play, demonstrating that “Cinema history and history cannot be separated from one another”: “In the field of tension between the history of cinema in the 1910s and gender history, a movement of emancipation takes place” (Schlüpmann 2013, 24).

Women were not only welcomed but were highly valued in the early motion picture industry, occupying positions from theatre staff to major writers and directors. Like Adams, despite their prominent positions, most did not publically identify themselves with the early feminist movement (Slide 1996, 1-3). Crucially Adams had also proven she had the talent, experience and determination to excel. Having starred in the commendable The Spirit of the Red Cross, she had also gained her family’s acceptance and support (Lake 1922, 51). From 1919 she was known publically not as “Clara” or “Peggy” but as Claire Adams, the “pet name” favoured by her family. Following The Spirit of the Red Cross, there were several projects to which she would dedicate her talents. One of these was a leading role in Romance and Brass Tacks (Martin Justice, 1918), “the new Flagg comedy starring pretty Peggy Adams, the famous Broadway beauty” (“An Unusual Comedy at Vining Theatre” 1918). The next motion picture Adams found herself starring in was another high-profile government sponsored film, also made for the war effort and proving remarkably popular. However, it also drew the kind of attention that few respectable young ladies from Winnipeg may have welcomed. Adams starred in one of the first major censorship scandals of the motion picture industry.

The End of the Road (Edward H. Griffith, 1919)

Young girls, thrilled with patriotism, sometimes fail to realize that the uniform covers all the kinds of men there are in the world; men of high ideals … or, in the worst instances, men who feel that their own physical appeal must be gratified, no matter who suffers. And so, through ignorance, through emotion, take steps which will lead to bitter regret.

(Katherine Bement Davis from Colwell 1998, 73)

Following The Spirit of the Red Cross and the additional projects with Flagg, there is evidence that Adams returned to theatre. In May 1918, she is named amongst the cast members of a patriotic play called Loyalty, written by George V. Hobart and staged at the Belasco Theatre in Washington DC. Belying both Adams’s preference for patriotic pieces as well as the public’s support for them, Loyalty is described as:

Something more than a mere dramatic entertainment … the play also carries a message of hope to a world suffering from the hardships and horrors of war (“Loyalty Will Make First Appearance At Belasco Tonight” 1918, 14).

The war may have been over but the horror continued on the home front. The “The Spanish Flu” evolved in the trenches of Europe from a moderately innocuous virus into something quite lethal. The appearance of the disease in New York City, borne by returning soldiers, was first reported in August 1918. The vius attacked the young and healthy, becoming more infectious and lethal with each mutation. Many industries, including motion picture companies and theatres, were unable to operate normally or in some instances were forced to close. Adams may have been forced to take time off or found work options cancelled or postponed. By the time the city recovered months later, it had lost 20,000 to 24,000 of its six million citizens. (Dominus, 2009). However, influenza was not the only infection rife amongst the troops.

In her essay, “The End of the Road: Gender, the Dissemination of Knowledge, and the American Campaign against Venereal Disease during World War 1”, Stacie A. Colwell provides a detailed account of the social and political context which gave rise to Adams’s next project (Colwell 1998, 44-82). As the US entered the war, the number of new recruits infected by venereal diseases appalled doctors conducting medical examinations.[23] As had been customary for generations, many young men of the time were engaging in unprotected pre-marital and extra-marital sex, ignorant or uncaring of the consequences. This practice was intensified during the heightened emotional climate of the war. Young women at the same time were becoming more personally independent, challenging the social order and demanding equal rights and freedoms to men. These included sexual freedoms.[24] They were unfortunately doing so within a vacuum of ignorance, a symptom of the perennial code of dual morality encouraging young men to explore their sexuality while young women were kept sexually innocent, unaware of their body’s functions and frailties. Many, deeply smitten, unknowing or unable to say no, found themselves bearing the physical and social consequences of sex with strangers or sweethearts they feared they’d never see again. The result was a perfect storm of ignorance, class-based bigotry and misbegotten morality, and Adams stepped right in the middle of it.

Eugenics, racism and class tensions exacerbated both the fear and apportioning of blame for the spread of venereal diseases (Schaefer 1999, 21). At a time when eugenics was still considered to be a viable scientific theory, some believed that the spread of sexually transmitted diseases (STDs) such as syphilis and gonorrhoea served a deliberate political or religious agenda.[25] Others saw the epidemic as an insidious weapon activated by the working masses, crowding into industrialised urban centres to breed out the richer, more refined classes (Colwell 1998, 72). The front line troops in this intimate means of class warfare were believed to be not simply prostitutes (who had a surprisingly low level of infection due to regulatory frameworks and inspections of brothels), but liberated “diseased and promiscuous” women seducing innocent men of means who then passed the infection onto their wives (Colwell 1998, 72). In turn, these ‘pure’ middle- and upper-class women would be driven mad, infertile, or both, by the ravages of the disease.[26] With many changes being made to the social order and a popular culture rife with inflammatory propaganda regarding invaders from abroad, there’s little wonder that this kind of imbalanced fear and paranoia infected the home front, like another disease.

The U.S. War Department, ever the pragmatists, saw the issue quite clearly – sick soldiers were bad for morale, an additional expense and made for a weakened army (Colwell 1998, 47). The challenge of addressing this issue fell to the War Department’s Committee on Training Camp Activities. This committee was confronting disciplinary issues relating to groups of young, independent and assertive New Women moving to towns and areas surrounding the training facilities. Schaefer implies that some were probably entrepreneurial prostitutes. Others were most likely “the khaki mad girls”, opportunists who simply loved a man in uniform and weren’t afraid to show it (Colwell 1998, 73). The troops themselves, predictably, were considered relatively blameless.

The momentum generated by the war to stem the spread of STDs provided an opportunity for diverse community organizations with similar concerns to affiliate with the War Department. A consortium of public health and volunteer organizations was incorporated under the banner of the American Social Hygiene Association (ASHA). Combined, they represented “a hybrid of social purity and sex education movements” (Colwell 1998, 46). These organizations pressed for a means of educating the population about the facts of life, the evils of STDs and pre-marital (or extra-marital) sex. Yes, it was bad for the War Effort. However for ASHA, the spread of venereal diseases was due to ignorance within a paradoxical culture. On the one hand young people had been loosened from the corset-like confines of the Victorian moral order, yet on the other were still bound by obscenity laws that made it illegal to disseminate medical or preventative information regarding STDs.[27] A “conspiracy of silence” reigned: people did not speak openly of such things and had few available sources of information (Schaefer 1999, 21). The volatility of the subject was amplified by the controversies surrounding the birth control movement. As Margaret Sanger and her supporters discovered, it was still considered obscene and illegal in the United States to distribute materials addressing the use of prophylactics and other forms of contraception.[28] Some considered it more socially repellent to talk of venereal disease in public than to actually contract a case and not mention it (Schaefer 1999, 21). ASHA concluded that a range of sensitive and educational, yet frank and fearless, vehicles were required, aimed at both men and women to address the facts of life and some of their indelicate consequences. As the Red Cross had realised before them, the military and their new allies embraced the power of motion pictures to educate and inform the public of America. A motion picture offered an additional bonus: In the darkness of a movie theatre, no one can see you blush.

Their strategy involved dividing the audience along gender lines. Fit to Win (Edward H. Griffith, 1919) was crafted to appeal to the needs and experiences of young male recruits.[29] The End of the Road (Edward H. Griffith, 1919) was aimed at young women who, out of curiosity, ignorance or early “girl power” gone awry, were finding themselves in physically and morally perilous positions. The film was to be screened with a pre-show lecture by a medical practioner in “Ladies Only” sessions at public cinemas. In order to make the subject matter more palatable for a “delicate” female audience, the educational aspects were wrapped up in a charming love story.[30] Conveniently, director Edward H. Griffith had been drafted into the War Department’s Committee on Training Camp Activities. Previously engaged by Thomas Edison Company, and with acting experience of his own, he played a major role in the production of several films supporting the war effort. During his time at Edison, Lt Griffith had made four films with Claire Adams. She was cast as Mary, the heroine of this sensitive story, opposite renowned actor Richard Bennett, who had enjoyed an expansive and celebrated theatre career before taking to the screen.[31]

A key figure within ASHA, Katharine Bement Davis was brought in to write The End of the Road. Davis described her contribution to the project as having “been most carefully worked out in consultation with physicians on the side of fidelity to medical fact, and with teachers as to the psychological effect” (Colwell 1998, 48). Davis had an established career in education, penal reform and public health, involving intensive research into the social causes of delinquency, the efficacy of reform programs for female prisoners, as well as surveys into the nature of women’s sexuality.[32] Her work had inspired John D. Rockefeller Jr. to establish the Bureau of Social Hygiene as part of his newly established foundation (Encyclopædia Britannica Online, 2014).

The film was produced under the supervision of the Surgeon General of the US Army, a protective political arm supporting the enterprise. Additional collaborators included the National War Work Council, the YWCA, Rockefeller’s Bureau of Social Hygiene and the Famous Players-Lasky Corporation (American Film Institute Catalogue, 2015). This was a high-powered group of influential stakeholders with an interest in the film’s success. There was more than the usual amount of pressure and expectation placed on the actress playing Mary, the leading lady in this high-stakes drama. The following synopsis is from the American Film Institute Catalogue:

Mary, whose mother has instructed her about love, marriage and sex, leaves her boyfriend Paul to become a nurse in a New York hospital. Vera, encouraged by her mother to marry a rich man, takes an apartment from a young millionaire who promises marriage but only gives her syphilis. Mary and her doctor treat Vera and show her examples of the ravages of the disease. Mary meets other suffering women: an Irish servant girl betrayed by a chauffeur who dies after her baby is born; a garment worker who contracted syphilis from a soldier’s forced kiss; the invalid wife of a wealthy man whose philandering caused her condition, the blindness of her child and the suicide of another of his conquests. Paul, about to enlist, suggests that he and Mary have sex before her goes. Disappointed in him, Mary also rejects the proposal of her doctor, but later in Europe after seeing the kind of man he is, she accepts him.

On all fronts, this was a brave role for Adams. Some of the scenes featured actual patients scarred and suffering from venereal diseases, filmed on location at the women’s wards of Blackwell’s Island, New York City.[33] In her role as the nurse Mary, Adams is seen conversing with these obviously frail patients, gently supporting them as they present for the camera.[34] Most of the film was shot at the Rockefeller estate at Pocantico Hills in Mount Pleasant, New York State (Colwell 1998, 63). As a resident of New York City at this time, it would have been difficult for Adams not to be aware of the campaigns for and against women’s suffrage and access to birth control, including the incarceration and exile of Margaret Sanger, all of which received widespread publicity between 1916 and 1918 (Banner 1974, 103). There is also the possibility that she may have had first-hand experience with the physical, social and psychological impacts of these diseases during her training and duties in the obstetrics ward at Grace Hospital in Detroit. That possibility becomes more probable when considering her contact with repatriated soldiers in Winnipeg. Perhaps this experience galvanised her willingness to participate, in spite of the risks. Her participation in The End of the Road reflects the influence of her family’s military and public service background as well as her own compulsion to help others. She would require the strength of the first and her faith in the latter to confront the controversy this film was about to generate.

Figure 5: Claire Adams as the nurse Mary Lee with a patient in The End of the Road (Edward W. Griffith, 1918). Source: UC Irvine Special Collections and Archives.

Figure 5: Claire Adams as the nurse Mary Lee with a patient in The End of the Road (Edward W. Griffith, 1918). Source: UC Irvine Special Collections and Archives.

The spark that ignited a furore was the sudden end of the war. This timing meant everything. Audiences and the industry had had enough of both war and sex.[35] The military units and programs that sponsored the film’s production were disbanded. Several key elements and references of the storyline were no longer relevant or threatened to offend peace-time sensibilities. Davis, however, successfully argued that the need to educate young adults, particularly young women, about their bodies remained, and that this justified public screenings. The project was permitted to “continue until a clear opinion has developed concerning the desirability or undesirability of its being shown through commercial channels” (Colwell 1998, 66). The ASHA gained control of the copyright of the film. The distribution and exhibition remained (for the time being) under the auspices of the Surgeon General, with safeguards in place to protect the film “from sensational exploitation”. Women’s groups and other officials were invited to preview screenings to judge its value and suitability for local audiences (Colwell 1998, 64).

The film premiered on 16 February 1919 in Syracuse, New York. There were fifteen hundred people at opening night and it went on to capacity crowds in a number of other cities (Schaefer 1999, 28). Most of the film’s key personnel were not there to see it. By opening night, Griffith had gone west. Davis sailed to post-war Europe pursuing her new role as General Secretary of Rockefeller’s Bureau of Social Hygiene. There was only one participant left with any skin in the game who made herself available to face down the critics and attend the film’s premiere. Colwell notes: “For the record, actress Claire Adams, who played Mary Lee, did travel to both Syracuse and Pittsburgh to defend the film she had starred in” (66). The New York Times reviewed the film in March 1919 and enthused that The End of the Road was “the most valuable motion picture of its kind yet produced.” The picture is not pleasant or euphemistic. Neither is the subject with which it deals. It is unpleasant, however, only to the degree necessary for force, and plain spoken only to the extent necessary for clearness. It is never morbid. One feels clean after seeing it (“Opening the Road” 1919). Acknowledging that audience responses would depend upon their own moral convictions, The New York Times also reassured readers: “No film would be effective without competent acting and directing and it is an important virtue of ‘The End of the Road’ therefore, that it had both in its making … Claire Adams and Joyce Fair, in the two leading female roles, were attractive in appearance and intelligent in their interpretation of their characters” (“Opening the Road” 1919).

Despite positive support in the mainstream media, critics of the film began protesting as soon as it was released. As The New York Times forewarned, the acceptance of the film was divided along the lines of the individual’s moral view and tolerance of the story’s unique premise: it was not just “patriotic prostitutes”, “army flappers” or “camouflage dames” responsible for the spread of STDs. The film dared suggest that decent middle class men played an active part.[36] The End of the Road was the first film to present the case that STDs did not always originate in women of the lower classes, demonstrating that “syphilis and gonorrhoea are equal opportunity diseases” (Schaefer 1999, 33). This perspective contributed to establishing a new set of social and political battlelines, just as the Great War ended.[37]

The pendulum swung back to a more conservative side of society, basking in victory and longing to normalise culture and behaviour (Schaefer 1999, 34). As a government sponsored project, The End of the Road drew intense criticism, more so than other sensational films that had preceded it (Schaefer 1999, 29). ­­­­­­­­­­According to critics in Moving Picture World, copies of The End of the Road and Fit to Win had also “fallen into the hands of individuals who are allegedly exhibiting it to mixed audiences composed of men, women, boys and girls” (“Association Goes After ‘Fit to Win’”, 1919, 1141). Prominent church figures denounced the film, led predictably by the Catholic Church (Schaefer 1999, 32). By July 16, 1919, it was banned in Philadelphia. National Association of the Motion Picture Industry (NAMPI), so successful in their production of The Spirit of the Red Cross, succeeded by the end of 1919 in having the film banned across the United States. This reflects what Löhrer describes as “a specific American moral panic at play” (Löhrer 2011).

Although the film could no longer be shown in the US, The End of the Road travelled abroad.[38] In May 1920 it was exhibited in Canada under the auspices of the Canadian National Council for Combating Venereal Diseases and the principals of local colleges. Demonstrating her ongoing commitment to the work, Adams reached out to her fellow Canadians with a personal message printed in The Vancouver Sun, subsequently reprinted across Canada. She is an unbowed advocate, certain “that every girl in Canada could see this play”:

The message it conveys is one that society must learn, and I feel that this powerful drama is the most wonderful method of telling the important story to girls everywhere. I played my part with that thought in my mind, and in my heart the feeling that at least to the best of my ability I was performing a real service to womanhood (“Message to Girls Through Film By Claire Adams” 1920).

The End of the Road found welcoming and respectful audiences even further afield. There is evidence that the film was used as an educative tool in military training campaigns as far away as Vladivostok (McMaster 2014, 5). Over the next decade The End of the Road was also shown extensively across Australia, where Adams would spend the second half of her life. Promoted in The Sunday Times in October 1920 with the US plan in place to segregate audiences along gender lines, the film opened at the Sydney Town Hall on Saturday 6 November 1920 and played for 5 weeks. The Evening News described how it had been shown the previous week at a private screening for representatives from the clerical, medical and legal professions, and was granted approval to be screened by the Minister for Womanhood. A search of Australia’s digitised newspaper archives shows that the film travelled through major metropolitan and regional centres across the country, from Muswellbrook NSW to Charters Towers QLD, Clare SA to Katanning WA. There is little criticism of the film evident: rather it received endorsement from civic leaders, educators and the press across the country.[39] Adams received rich praise. In Melbourne it was shown at the Palace Theatre on Bourke Street where the Table Talk reviewer declared in March 1921, “Nothing better has been shown on the screen than the perfectly moulded features of Claire Adams.” When The End of the Road reached Hobart in June 1921, The Mercury reported that over 5,000 had already seen it in Launceston. The reviewer from The Advertiser in Adelaide, who had seen the film at the Adelaide Town Hall, enthused in July 1921 that “The picture defies description. There is a touch of genius in it. The screen has never disclosed a purer or more impressive lesson.” The following week, the same paper discussed how several members of the public showed great interest in Adams, enquiring after her background and identity. They make much of her youth, talent and beauty, but pay particular attention to her patriotic service during the war and her nursing experience at Grace Hospital in Detroit.

The film continued across Australia until July 1928, when The Goulburn Evening Post reported that the reels had been stolen in Broken Hill. By that time, however, Adams had retired from her successful Hollywood screen career. One decade later, Claire Adams migrated permanently to Australia, stepping off the ship in Melbourne on Valentine’s Day in 1938, where there may still have been more than a few moviegoers who remembered her work. There is evidence to suggest that Claire Adams Mackinnon had a copy of both The Spirit of the Red Cross and The End of the Road with her when she came in Australia. It is not known, however, if they were shown beyond the bounds of her new home, or if those who saw them during one of her movie nights had any idea of the films’ impact during and in the aftermath of the First World War.[40]

Figure 6: Claire Adams as Justyn Reed, the fiancé of Jim Apperson, played by silent screen idol John Gilbert in The Big Parade (King Vidor, 1925). Image ©Warner Bros. Source: Author’s private collection.

Figure 6: Claire Adams as Justyn Reed, the fiancé of Jim Apperson, played by silent screen idol John Gilbert in The Big Parade (King Vidor, 1925). Image ©Warner Bros. Source: Author’s private collection.

Adams’s performance in The End of the Road drew the attention of producers in Los Angeles and New York, where they were deep in debate regarding proposed censorship measures within the motion picture industry. In 1920, independent producer Benjamin B Hampton, who had also seen her work in The Spirit of the Red Cross invited her to Hollywood to star in his productions (Lake 1922, 51). They would marry in 1924. By the time she retired in 1928 she had starred in over forty feature films alongside some of the most popular and defining artists of the era. These included Tom Mix, Rin Tin Tin, Jean Hersholt, Lon Chaney and Clara Bow. Adams’s also appeared opposite John Gilbert in his break through role in The Big Parade (King Vidor, 1925), noted in the AFI catalogue as being “frequently described as the most successful silent film of all time”. Adams played Justyn Reed to Gilbert’s doughboy Jim Apperson. Justyn is a blithe ingénue, somewhat reminiscent of Adams’s flirtatious character Nina from The Man-eater, gaily encouraging her fiancé to enlist so she can see how handsome he’d look in uniform. These characters, both Nina and Justyn, could well have been constructed from characteristics of the sexually liberated yet innocent/ignorant young women who were the target audience for The End of the Road. As such, The Big Parade is a strangely coalescent bookend for her career, which practically began and ended with blockbusting war films.

Conclusion.

The value of the Claire Adams films discussed here is lost when they are removed from their historical context. They are very much a product and reflective of the era in which they were produced, which was both defining and tumultuous. The confluence of war, the women’s independence movement and the development of the motion picture industry offered women a new degree of independence and a more active role within their own lives as well as within their culture, both as consumers and producers. This relatively small period from Adams’s career encapsulates those opportunities, as well as the threats and challenges available to women in general and within the motion picture industry in particular. She is a mediator between the realities and the fictions; a nurse playing a nurse addressing current issues both on and off-screen, lending more than a modicum of verisimilitude to the productions. Her characters in The Spirit of the Red Cross and The End of the Road were strong female protagonists, created to inspire audience members and advocate for action, encouraging all women to change their lives and in doing so play an active role in changing their world.

Amidst the international commemorations of the centenary of The Great War, it is timely to assess Adams’s life of service in front of the camera and behind the scenes, not only in Australia, but also for our allies in the conflict – Canada and the United States of America. She was, after all, a proud citizen of each country at different stages of her life. That her achievements have been forgotten is also symptomatic of the times. The role of women in the motion picture industry has been undervalued and overlooked for most of the previous century, so the “loss” of Claire Adams from film history is not unusual. The scale of public awareness generated around the superstars of Silent Hollywood, fed by the scandals, stereotyping and celebrity culture that engulfed the industry, has diminished the memory of her career, especially if it is measured and evaluated in terms of current and historic memory. My biography of Adams (in progress) explores her significance, of the difference she made as an actress and as a private citizen, and how she never lost the sense of public service first demonstrated in the choice of roles discussed here. Indeed, Adams continues to support community causes, including the Australian Red Cross, via a substantial trust established after her death in 1978.[41] By avoiding the superficiality and pressures of the studio system, Adams may have missed out on Hollywood immortality. However, her personal and professional choices demonstrate that she was not overly concerned by the vagaries of popular opinion. Her priorities were always much closer to home.

 

 

DEDICATION:

Many people in Australia, the US and Canada have contributed to this article and my research into the life of Claire Adams Mackinnon. However, I would like to dedicate this article to Mr Roger Mayer, who passed away in March 2015. Some will remember him as the recipient of the Jean Hersholt Humanitarian Academy Award in 2005. Others will for his longstanding leadership of the U.S National Film Preservation Foundation. I will always remember him as a kind gentleman who upon hearing of my interest in Claire Adams insisted I pursue it with the words “Every piece of the story is worth saving”.

 

Bibliography:

American Film Institute Catalogue. “The End of the Road.” Accessed 29 August 2014. http://www.afi.com/members/catalog/DetailView.aspx?s=&Movie=15366

“An Unusual Comedy at Vining Theatre.” Ashland Tidings. Tuesday 31 December 1918. 1. Library of Congress.

Ancestry.com, “Census records for District 15, Province of Manitoba. 1916. Census of Manitoba, Saskatchewan and Alberta. 51.”, Canadian Census Records. 1916. Accessed 27 October 2009.

Ancestry.com, List or Manifest of Alien Passengers Arriving Port of Michigan, June 1915, Sheet No. 35, Border Crossings: From Canada to U.S., 1895-1956, accessed 27 October, 2009.

 Ancestry.com, Petition for Citizenship, Claire Adams Hampton, United States of America, No. 3441, 4 February 1931, U.S. Department of Labor, accessed June 2014.

“Association Goes After “Fit to Win””. 1919. The Moving Picture World. May 24. 1141. Margaret Herrick Library, Academy of Motion Picture Arts and Sciences.

Banner, Lois W. 1974. Women in Modern America: A Brief History. New York: Harcourt Brace Jovanovich, Inc.

Barnaby Lee. The American Film Institute Catalogue. Accessed 29 August 2014. http://www.afi.com/members/catalog/DetailView.aspx?s=&Movie=2108

A Brief History of the American Red Cross, The American Red Cross, accessed 19 August 2014, http://www.redcross.org/museum/history/ww1a.asp

The Canadian Red Cross. 2014. “About the Canadian Red Cross”, Accessed 15 August 2014, http://www.redcross.ca/who-we-are/about-the-canadian-red-cross/history-frequently-asked-questions

“Claire Adams, Starring in ‘The End of the Road’ Is Real Western Canada Girl”, 1920. Saskatoon Phoenix. June 19. Library of Congress.

Colwell, Dr Stacie A., “The End of the Road: Gender, the Dissemination of Knowledge, and the American Campaign against Venereal Disease during World War I.” In The Visible Woman: Imaging Technologies, Gender and Science, edited by Paula A. Treichler, Lisa Cartwright and Constance Penley, 44-82. New York: New York University Press, 1998.

Dall’Asta, Monica and Duckett, Victoria. 2013. “ Kaleidoscope: Women and Cinematic Change from the Silent Era to Now.” In Researching Women in Silent Cinema: New Findings and Perspectives. Edited by Monica Dall’Asta, Victoria Duckett, Lucia Tralli. Women and Screen Cultures. Vol 1. University of Bologna in association with the Victorian College of the Arts, University of Melbourne and Women and Film International. 8-11.

Deer Lodge Centre. 2014. “History.” Accessed 19 August 2014. http://www.deerlodge.mb.ca/aboutHistory.html

 

Dominus, Susan. 2009. “In 1918 Flu Outbreak, a Cool Head Prevailed.” The New York Times. May 1.

“Edison Releases Flagg Social Satire Series.” 1917 Motion Picture News, Vol 16. No 26. December 29. 4591. The Seaver Centre for Western History Research, Natural History Museum of Los Angeles County.

Encyclopædia Britannica Online, 2014. “Katharine Bement Davis”. Encyclopædia Britannica Inc.. Accessed 17 Aug. 2014 http://www.britannica.com/EBchecked/topic/152802/Katharine-Bement-Davis

“The End of the Road.”1921. The Advertiser. 11 July. Retrieved from TROVE, National Library of Australia. http://nla.gov.au/nla.news-article73372048

“The End of the Road.” 1921. The Advertiser. 16 June. Retrieved from TROVE, National Library of Australia. http://nla.gov.au/nla.news-article35249538

“The End of the Road.” 1921. The Mercury. 8 June. Retrieved from TROVE, National Library of Australia. http://nla.gov.au/nla.news-article23515162

“The End of the Road.” 1921.The Northern Daily Leader. 10 February. Retrieved from TROVE, National Library of Australia, http://nla.gov.au/nla.news-article104816157

“End of the Road.” 1920. Sunday Times. 31 October. Retrieved from TROVE, National Library of Australia, http://nla.gov.au/nla.news-article120516928

““End of Road” Barred.” 1919. The Moving Picture World. July. Margaret Herrick Library, Academy of Motion Picture Arts and Sciences.

“End of the Road Barred.” 1919. Variety. 18 July. Margaret Herrick Library, Academy of Motion Picture Arts and Sciences.

“The End of the Road: a Movie of Disease.” 1920. Evening News. 3 November. Retrieved from TROVE, National Library of Australia, http://nla.gov.au/nla.news-article117303505

Goldbeck, Willis, “My Lady Claire”, Motion Picture Classic, 1920, from Mooramong Collection, National Trust of Australia (Victoria).

Hampton, Benjamin B., History of the American Film Industry from its beginnings to 1931, Dover Publications, Inc., New York, 1970, ed. Richard Griffith, Curator Emeritus, The Museum of Modern Art Film Library.

Hampton, Benjamin B., A History of the Movies, Covici, Friede, New York, 1931.

Hellier, Donna, Endnotes to “Social History Report” included in O’Connor, John and Thurley, Mooramong Buildings and Structures: Conservation Analysis Report for the National Trust of Australia (Victoria), July 1989.

“Items of Interest”, Goulburn Evening Post. Tuesday 17 1928, Retrieved from TROVE, National Library of Australia, http://nla.gov.au/nla.news-article102527976

Jackal, Susan and Millette, Dominique. Revised 2015. “Women’s Suffrage”. Historica Canada. Accessed 12 June 2015. http://www.thecanadianencyclopedia.ca/en/article/womens-suffrage/

Katz, Esther. 2000. “Margaret Sanger.” American Dictionary of Biography Online. Accessed 3 February 2015. http://www.anb.org/articles/15/15-00598.html )

Lake, J. Marion. 1922. “A Young Lady in Earnest.” Motion Picture Classic. May.

Letourneau, J. A. Rodger, “Kennedy, William Nassau”, Dictionary of Canadian Biography, University of Toronto/Université Laval, 2003. Accessed August 13, 2014. http://www.biographi.ca/en/bio/kennedy_william_nassau_11E.html.

Löhrer, Gudrun. Conference Report. Communicating Good Health: Movies, Medicine and the Cultures of Risk in the Twentieth Century. 26-27 May 2011. Brocher Foundation. Geneva/Hermance, Switzerland. Accessed 18 August 2014.

Louisiana Film History by Parish, Hollywood on the Bayou. 2015. http://www.learnaboutmovieposters.com/newsite/louisiana/Parishes/Orleans.asp

“Loyalty Will Make First Appearance At Belasco Tonight.” 1918. The Washington Times. 26 May. Library of Congress.

The Manitoba Historical Society, “William Herbert Adams”. Last modified 24 December 2009. http://www.mhs.mb.ca/docs/people/adams_wh.shtml

Marshall, Edward. 1914. “New York’s First Woman Commissioner of Correction”, The New York Times, 11 January.

Maxwell, Virginia. 2000. ‘Mackinnon, Claire Adams (1896–1978)’. In Australian Dictionary of Biography, National Centre of Biography, Australian National University, Accessed 15 August 2014. http://adb.anu.edu.au/biography/mackinnon-claire-adams-10996/text19553,.

McMaster, Christopher T., “The International Military Police and the Allied Intervention in the Russian Civil War”, Student Pulse, Vol 6, No. 04, Retrieved 18 August 2014. http://www.studentpulse.com/a?id=891

“Message to Girls Through the Film By Claire Adams”, The Vancouver Sun, May 21, 1920. Proquest.

Measuring Worth. Williamson, Samuel H. “Purchasing Power of Money in the United States from 1774 to Present.” 2013. URL: www.measuringworth.com/ppowerus/

“Opening the Road.” 1919. The New York Times. March 2. Proquest.

“Park Avenue at 40th Street, 1920s”. Digital Murray Hill from the Mina Rees Library. Accessed 2 December, 2011. http://library.gc.cuny.edu/murrayhill/items/show/271

Petition for Citizenship, Claire Adams Hampton, United States of America, No. 3441, 4 February 1931, U.S. Department of Labor, from Ancestry.com, accessed June 2014.

 Prospectus of the Grace Hospital Training School for Nurses, 1888. Grace Hospital Collection Papers 1880-1978, Walter P Reuther Library, Wayne State University.

Quiney, Linda J. 1998. “Assistant Angels: Canadian Voluntary Aid Detachment Nurses in the Great War.” Canadian Bulletin of Medical History. 02/1998. Accessed 15 August 2014.

“Recollections of ‘Girls You Know.’”1918. Motion Picture Magazine, V15, No 5, June, 78.

Rozario, Kevin. 2003. “Delicious Horrors”: Mass Culture, The Red Cross, and the Appeal of Modern American Humanitarianism.” In American Quarterly, Vol. 55, No. 3 (September). American Studies Association, The Johns Hopkins University Press, 417-454.

Schaefer, Eric. 1999. Bold! Daring! Shocking! True! A History of Exploitation Films, 1919-1959. Durham: Duke University Press.

Schlüpmann, Heide. 2013. “An Alliance Between History and Theory.” In Researching Women in Silent Cinema: New Findings and Perspectives. Edited by Monica Dall’Asta, Victoria Duckett, Lucia Tralli. Women and Screen Cultures. Vol 1. University of Bologna in association with the Victorian College of the Arts, University of Melbourne and Women and Film International. 13-26.

Slide, Anthony. 1996. The Silent Feminists: America’s First Women Directors. London: The Scarecrow Press, Inc.

Spartacus Educational. “James Montgomery Flagg.” Accessed 14 August 2014. http://spartacus-educational.com/ARTflagg.htm

“Spirit of the Red Cross.” 1918. The New York Times, 21 April. Proquest.

“Splendid Work of American Red Cross Society Graphically Told in Series of Single Reel Motion Pictures Now Ready for Screen”. 1918. Exhibitors’ Trade Review. December 7. Vol 5. No. 1.

Stamp, Shelley. 2012. “Women and the Silent Screen”. In The Wiley-Blackwell History of American Film. Edited by Cynthia Lucia, Roy Grundman, and Art Simon. University of Santa Cruz. Accessed 10 January 2015.

Theatre advertisements. 1919. The Youngstown Daily Vindicator. 4 April. Library of Congress.

“V.A.D Hospitals in Northumberland and Durham 1914-1918.” Donmouth Local History Society. Accessed 18 February 2015 http://www.donmouth.co.uk/local_history/VAD/VAD_hospitals.html

“What People are Saying and Doing.” 1921. Table Talk. 10 March. Retrieved from TROVE, National Library of Australia. http://nla.gov.au/nla.news-article146718304

Will, Beverley. 1978. “Gallant and Romantic were the days of Claire Adams.” Green Place. October. Mooramong Collection, National Trust of Australia (Victoria).

The World War 2 US Medical Research Centre, http://www.med-dept.com/articles/venereal-disease-and-treatment-during-ww2/ Accessed 29 January 2015.

Unpublished sources:

  • Hone, Geoff. (Chair of the Scobie and Claire Mackinnon Trust.) Email to author. June 2014.
  • Miley, Nancy. Adams Family History, Unpublished Manuscript. Author’s private collection.
  • Schmeling, Kathleen Emery. (Archivist) Grace Hospital Archives, Wayne State University. Email to author, May 21 2014.
  • Speed, Dr Lesley. “‘In the best film star tradition’: Claire Adams and Mooramong”, Screening the Past, due for release 2015. http://www.screeningthepast.com/
  • Sweet, Jill and Stephens, Barbara Adams. Oral History Transcript, Author’s private collection.

Filmography:

  • Caneva, Lina, 2009, Mooramong – Private Hollywood, Caneva Media Productions.
  • Eaton, Jack, 1918, The Man-eater, Thomas A. Edison, Inc.
  • Griffith, Edward H., 1917, Scouting for Washington; a tale of the Revolution, Thomas A. Edison, Inc.
  • Griffith, Edward H., 1917, Shut Out in the Ninth, Thomas A. Edison, Inc.
  • Griffith, Edward H., 1919, The End of the Road, American Social Hygiene Association, Famous Players-Lasky Corp & US War Department.
  • Vidor, King, 1925, The Big Parade, Metro-Goldwyn-Mayer.

 

Notes:

[1] Adams’s petition for US citizenship document confirms that she was a subject of the British Dominion of Canada and that her last foreign address was in Toronto. From there she entered the US permanently at Detroit on 12 June, 1915. She first declared her intention to become a US citizen in September 1922, two years before she married Hampton. Her petition for citizenship was granted in Poughkeepsie on 4 February 1931, having resided in Pawling, NY since 1 September 1928.

[2] Willis Goldbeck in Motion Picture Classic (1920) described Adams as “instinctively Britishly reserved, a person whom one cannot hope to know in a day, or a month.”

[3] As of this date, I have yet to find Claire Adams on the cover of either fan or trade magazines, though several profile interviews and letters to editors indicate she had a fan base. Without a studio publicity department behind her, and as part of a company that promoted all-star casts, she was able to enjoy a more private life and manage her own profile accordingly, hence my concept of her as an independent artist.

[4] Originally published as A History of the Movies, by Covici, Friede, New York, 1931, Hampton’s work was reissued in 1971 as History of the American Film Industry from its beginnings to 1931, edited with an introduction by film historian Richard Griffith.

[5] In his introduction to the second edition of Hampton’s book, Griffith asserted “That it is the best history of the movie business to date there can be no doubt’, and positions it in terms of authenticity and objectivity above Terry Ramsey’s A Million and One Nights (1926) and Lewis Jacobs’ The Rise of the American Film (1939).

[6] As described by Monica Dall’Asta and Victoria Duckett in “Kaleidoscope: Women and Cinematic Change from the Silent Era to Now.” 2013, 8.

[7] At only fifteen years of age, Gerald ran away to join the Scottish Highlanders, but was dragged back to London by his Great Aunt Mabel Adams with whom he was staying. As soon as he was old enough he enlisted, but had only just completed training when the war ended. He never saw any action on the front. Nancy Miley,“Gerald Drayson Adams (1900-1988)” from Adams Family History Notes, Unpublished Manuscript.

[8] This prospectus was written late in the 19th century. The Grace Hospital School of Nursing (GHSN) offered two years of vocational (if somewhat rudimentary) instruction during which time students would be under constant supervision, on and off duty. Students were permitted one afternoon off per week and were encouraged for the sake of their own health to spend one hour each day in the open air. They were allowed two weeks leave per year, one evening off per week and required permission from the school’s Principal to be out later than midnight. In return, the students would receive training in elements of hospital work, namely the dressing of wounds and burns, applications of fomentations and poultices, making beds and methods for avoiding bedsores.

[9] Adams’s name on all official documents was her birth name, Beryl Vere Nassau Adams, but this does not show up on the list of graduates either.

[10] According to the Red Cross Canada website, “Red Cross nurse” was a term applied to anyone volunteering in a hospital in any capacity, including letter writing and visiting. Millions of young women contributed to this international effort and there was simply not the capacity to establish or maintain complete records of every volunteer. “About the Canadian Red Cross”, Accessed 15 August 2014, http://www.redcross.ca/who-we-are/about-the-canadian-red-cross/history-frequently-asked-questions

[11] A New Set of Needs, from the website of Veterans Affairs Canada, describes the inadequate hospital infrastructure, and venues converted to repatriation hospitals following England’s requests to the Dominions to take care of their own wounded. Accessed 11 January 2015, http://www.veterans.gc.ca/eng/steannes-hospital/about-us/history

[12] According to the American Film Institute Catalogue entry, Barnaby Lee was probably not widely released. http://www.afi.com/members/catalog/DetailView.aspx?s=&Movie=2108

[13] Scouting for Washington, Wild Arnica and Shut Out in the Ninth were viewed by the author at the Library of Congress, Washington D.C.

[14] Flagg asserted that his Uncle Sam became “the most famous poster in the world”. As quoted on the American Treasures of the Library of Congress website, accessed 13 February 2015, http://www.loc.gov/exhibits/treasures/trm015.html

[15] The Man-eater was viewed by the author at the Library of Congress, Washington D.C.

[16] The other lead roles in the “Girls You Know” series were played by Dorothy Wallace, Mary Arthur, Martha Mansfield and Peggy Hopkins. According to IMDb, Hopkins had appeared in 2 Columbia productions. Dorothy Wallace had one previous film credit. Martha Mansfield had four previous motion picture credits and seemed destined for a successful career. In 1920 she starred opposite Lionel Barrymore in Dr Jekyll and Mr Hyde (John S. Robinson, 1920). Mansfield died in horrific circumstances in 1923 after her period costume caught fire on set. See IMDb http://www.imdb.com/name/nm0543806/bio?ref_=nm_ov_bio_sm

[17] Banner suggests that it was considered “a disgrace for a woman’s name to appear in public print” though this began to change before the war (20).

[18] According to Nancy Milley, her father Claire’s Uncle, Ernest Adams and her Grandmother, did not approve of Claire’s profession, even into the 1920s when she was at the height of her career. “Ernest Dupin Adams” from Adams Family History Notes, unpublished manuscript, author’s private collection.

[19] Several scenes were shot in the Jackson Barracks in New Orleans, which was in use as a training and processing facility for the United States Army. This suggests that recruits may have been used as extras in the film, perhaps to add a layer of verisimilitude to the scenes of battle and troop movements. From website Hollywood on the Bayou, Louisiana Film History by Parish, http://www.learnaboutmovieposters.com/newsite/louisiana/Parishes/Orleans.asp

[20] The Red Cross War Council subsequently established a “Bureau of Pictures” in early 1918 “to tell upon the screen the splendid story of the Red Cross”. These single reel shorts featured such titles as “Broken Lives”, “Victorious Serbia” and “Russia: a Land Worth Saving”, reflecting their ongoing work in Europe and with veterans up to and following the Armistice. From “Splendid Work of American Red Cross Society Graphically Told in Series of Single Reel Motion Pictures Now Ready for Screen”. 1918. Exhibitors’ Trade Review. December 7. Vol 5. No. 1.

[21] In the El Paso Herald, for example, the proprietors of the Grecian Theatre placed a large advertisement, paid for by another local business, outlining a special “Red Cross War Fund Benefit”, from which they would donate all proceeds from the evening’s entertainment to the cause. Patrons were encouraged to “make your quarters jingle” at the box office as “Lives Over There” depended on their support. Wednesday 15 May 1918, 5.

[22] The release of the film was also accompanied by a nation wide door knock appeal (American Red Cross, 2014).

[23] The World War 2 US Medical Research Centre estimates that during the Great War, venereal disease “had caused the Army lost services of 18,000 servicemen per day.” http://www.med-dept.com/articles/venereal-disease-and-treatment-during-ww2/ Accessed 29 January 2015.

[24] As described by Banner, early feminists saw male sexuality as being at the heart of female oppression. Even before the war, early feminist Inez Milholland wrote that “we are learning to be frank about sex … and through all this frankness runs a definite tendency toward an assault on the dual standard of morality and an assertion of sex rights on the part of women” (116).

[25] According to Schaefer, “Discourses on venereal diseases and eugenics were so tightly intertwined as often to be inseparable” (21).

[26] The irony is that throughout the nineteenth century, prostitutes were seen as a necessary evil in order to protect the virtue of pure women. The North American sex industry operated in relatively structured or semi-licensed conditions where the workers were regularly checked for STDs (Banner 1974, 76). The change came in the early twentieth century when sexually active young women identified sex with strangers as a possible way out or momentary escape from the drudgery of their working class lives. If they accepted payment for sex, they could earn two to three times more than they could as a domestic servant or salesgirl (Banner 1974, 81).

[27] Schaefer laid out the confluence of concerns that surrounded the spread of STDs in American society, which included Eugenics, Industrialization, Abstinence and Prohibition. The constituents incorporated under the ASHA banner represented most of these concerns. Progressives were concerned about the political and social boundaries being blurred by the urban, industrial and technological developments of the early Twentieth Century (18 – 23).

[28] According the Katz, Sanger had already experienced exile from the US in 1914 when in 1916 she was arrested and imprisoned for opening the nation’s first birth control clinic. In 1917 she had also made a motion picture titled “Birth Control” which had been confiscated by New York authorities.

[29] Fit to Win was shown at training camps where a lecturer would be available to explain some of the finer points of the storyline and address any medical questions.

[30] The film’s writer, Katherine Bement Davis, had already conducted extensive research into the possible impact of sex education on young women, which contradicted prevailing Freudian based theories that too much knowledge of sexuality and its consequences could permanently damage young women (Colwell 1998, 60).

[31] In 1914 Bennett made his movie debut with Damaged Goods, a film adapted from the theatre script that “pictures the terrible consequences of vice and the physical ruin that follows the abuse of moral law.” Bennett had married his co-star from Damaged Goods, Adrienne Morrison, with whom he had three daughters, two of which would go on to major Hollywood careers: Joan and Constance Bennett.

[32] Banner describes Davis as a leading figure of the Progressive movement, instigating major studies and reforms in the fields of urban poverty, women’s prisons and sex education (Banner 1974, 98). The New York Times ran a major profile of Davis when she was named New York’s first female Commissioner of Correction in January 1914. See Edward Marshall’s “New York’s First Woman Commissioner of Correction”, The New York Times, 11 January 1914.

[33] The location where these scenes were shot is mentioned in the notes section for the catalogue entry for The End of the Road in the American Film Institute Catalogue accessed 29 August 2014, http://www.afi.com/members/catalog/DetailView.aspx?s=&Movie=15366. Schaefer describes how many of the more confronting scenes of open wounds were cut in order to please the censors and keep the film in theatres, but some scenes were later reintroduced by exploitative distributors (29-30).

[34] The End of the Road was viewed by the author at the Library of Congress, Washington D.C.

[35] The issues facing the film were compounded by the film industry’s wish to appear respectable and beyond moral reproach in the face of a growing amount of sex scandals and exploitation films (Schaefer, 30). There was also an acceptance of sex education taking place within schools, making the role of sex education films in public cinemas largely redundant (Colwell, 69).

[36] According to Colwell (71), there were around ten other films addressing the spread of STDs, however they depicted women as the source of infection and transmission. The End of the Road differs significantly by drawing attention to the complicity of men.

[37] Schaefer notes that Damaged Goods (Thomas Ricketts, 1914) also starring Richard Bennett was the first to have established this premise on film. Damaged Goods reinforced the claim that venereal diseases were the scourge of the lower classes and inflicted upon young men of means and family during moments of weakness caused by drunkenness or deliberate seduction by fallen women from the lower classes (23).

[38] The copy viewed by the author in the Library of Congress was in Dutch, indicating it was intended for audiences in the Netherlands and other Dutch-speaking colonies.

[39] This section could not have been conducted without the use of Trove, the National Library of Australia’s online database of Australia’s newspapers. The articles directly referenced are noted in the above reference section. http://trove.nla.gov.au/newspaper/result?q=The+End+of+the+Road

[40] The Mooramong Buildings and Structures: Conservation Analysis Report for the National Trust of Australia (Victoria) of July 1989 contains a list (Appendix A) of 48 motion pictures “compiled from film reels held at Mooramong”. The fate of these films is unknown.

[41] According the Geoff Hone, the Scobie and Claire Mackinnon Trust have donated approximately AU$2 million to the Royal Children’s Hospital as well as other significant amounts over the years to the Australian Red Cross.

 

Bio: Heather L. Robinson is a Research Associate & PhD Candidate in the School of Humanities and Creative Arts, Flinders University. She is also an Honorary Research Associate (History) at the Los Angeles Natural History Museum.

Days of YouTube-ing Days of Heaven: Participatory Culture and the Fan Trailer – Kyle R. McDaniel

Abstract: This study analyzes the aesthetic content and user-generated feedback of fan-appropriated film trailers exhibited in on the Internet. The aim of this research is to gauge participatory culture’s involvement in the transformation of promoting archival motion pictures on the Internet. This research study looks to fan trailers as unique media entities that exist as visually empowered narratives created through specific acts of fandom. Specifically, this study investigates the audiovisual and discursive elements of competing trailers for Terrence Malick’s Days of Heaven (1978). The findings suggest that fan trailers are capable of generating myth and nostalgia for aging motion picture properties through user-generated acts. The broader goal of this project is to understand the relationship between participatory film cultures and studio-controlled motion picture content available on video streaming and sharing media channels.


Fig. 1: The memorable and Biblically referential swarm of locusts in the film Days of Heaven.

Fig. 1: The memorable and Biblically referential swarm of locusts in the film Days of Heaven.

Introduction: Trailers at a Turning Point

A YouTube video by an unknown director can suddenly blow up on the marketplace, and there will be three studios bidding for it. (Without having yet met the director!)…Maybe execs are busy watching YouTube instead of hearing pitches. Our work is virtual.   

-Lynda Obst, Sleepless in Hollywood (2013, 27).

In April 2014, an online user released a high-definition film trailer on YouTube for David Fincher’s forthcoming thriller, Gone Girl (YouTube 2014a). Several hours after the trailer’s debut, an impressive 186,000 fans had accessed the content with 276 of that number contributing written feedback to the message board on the webpage. While film fans were sharing interest and excitement for the trailer on YouTube, News Corp., the media entity that financed Gone Girl through 20th Century Fox, perceived a threat of digital piracy. The following day, the conglomerate removed the trailer and the fan commentary. In the absence of this content, News Corp. left a statement reading, “FOX has blocked [the trailer] on copyright grounds” (YouTube 2014b). This incident is representative of the contemporary state of affairs between media conglomerates with a controlling interest in motion pictures and film fans in online spaces. The presence of film trailers on the Internet presents a specific set of issues for both parties as well, especially in relation to film marketing and promotion, in addition to content ownership and control over copyright.

This study engages with how film fans interact with once-profitable motion picture properties through fan trailers on the Internet. Here, the fan trailer is defined as the act of re-editing and re-exhibiting abridged film content through online channels. Fan trailers are realized through specific and largely collective acts of user-participation, and have the potential to revitalize interest in aging film properties. This article explores the audiovisual and content-related aspects of fan trailers in comparison to a distributor-owned trailer for Days of Heaven (Terrence Malick 1978). Furthermore, the feedback or commentary on message boards is also investigated as part of this research project to locate how such discourse speaks to the collective memory of Hollywood archives. In order to understand the issues surrounding the emergence and popularity of the broad spectrum of Internet trailers, this study looks to literature on the relationship between the evolution and of fan involvement with digital cinema and new media, as well as scholarship on the history of film trailers and film promotion and advertising. The findings from this article suggest that fan trailers play a crucial role in continuing the lifespan of aging Hollywood properties or archival films. The proliferation of fan trailers through video streaming and sharing websites as well as the message board commentary suggests that fan participation is instrumental to building relationships between film and viewer. In turn, participatory cultures that interact with older film titles in online channels incorporate aspects of their public and private selves as part of this creative process. The following research questions are designed to further explore this relationship between film fans inhabiting online spaces and the evolving state of fan trailers in digital cinema: What are the content-related (i.e., audiovisual) similarities and differences between the distributor-controlled, official trailer and the fan trailers under study? And what role(s) does user-generated commentary or feedback play for these trailers?

Fig. 2: A black-and-white still of Terrence Malick on the set of the film.

Fig. 2: A black-and-white still of Terrence Malick on the set of the film.

Film Promotion in the Digital Age: New Strategies, New Rules

For much of the 2000s, Hollywood was reluctant to promote film content through online channels for fear of losing theatrical and home video revenue (Sickels 2011a). The film industry seemed confused by the ever-growing presence of the Internet and related online technologies for film exhibition. But to effectively reach a global audience, the studios and their parent media conglomerates were eventually forced to adapt to the changing media landscape. As Sickels (2011) stated: “Deals with Netflix and the like are only going to delay the inevitable…Audiences don’t want to wait, and they certainly won’t when their only reason for having to do so is an artificial time structure concocted by the studios…”(145). By the second decade of the century, the industry’s fears had become a reality, with on-demand film and television viewing radically altering the industry.

Scholars have pointed to the different complexities of film marketing in the digital age and the associated challenges for the U.S. film industry (e.g., Cunningham and Silver 2013). In Perren’s (2010) words, “A wide range of economic, cultural, political, and formal factors are at play; different entities have distinctive stakes in online distribution” (77).  In other words, films with a greater potential to appeal to a global audience receive preferential treatment from media conglomerates, as well as promoters, marketers, and distributors. With video-on-demand (VOD) revenue climbing steadily since 2010, the studios are looking to different methods for advertising motion pictures beyond the more traditional formats, which includes one-sheets of film posters and theatrical and television spots (Roxborough 2013). Film trailers on the Internet are a viable option in this evolving landscape. The Internet Movie Database and YouTube are the most frequently visited websites supporting online film trailers, with both entities supporting numerous trailers for new releases and older Hollywood titles. In effect, the spectrum of film trailers on the Internet presents a number of potential issues for the film industry. Trailers, historically controlled by studios for advertising and publicity purposes, are increasingly pirated by outside entities. One scholar argues that film industry insiders are the ones largely responsible for leaking studio-controlled content online, with the availability of illegal anti-encryption and watermarking software to bypass copyright restrictions playing a role as well (Bettig 2008, 200-201). Since the release of the DVD De-Content Scramble System (DeCSS) in 2002, film content has been descrambled and decoded for public access and use, despite the studios efforts to control motion picture content (Litman 2002).

Film fans, however, have argued that such laws overwhelmingly favor those with a financial stake in motion picture properties, thereby inhibiting individual and collective acts of creative expression (Boyle 2008). As such, studio-backed restrictions have resulted in more frequently cited instances of pirated motion pictures as well as an upsurge in websites devoted to streaming and downloading studio-owned film content (Sterbenz 2014). Scholars and journalists reporting on the film industry have addressed some of these issues in relation to film trailers. For instance, Rothman (2014) discussed how theatrical trailer standardization discourages user interactivity. Tolson (2010) reported that fan participation with film content suggests an increase in technological “play” that disrupts the traditional model of media production to consumption. Others have looked at how trailer “mobility” is encouraged in a cross-platform media environment, and the effects of contemporary trailer length and message on the viewer (see Franich 2013; Johnston 2008). While many of the issues surrounding film promotion in online spaces remain unanswered, trailers continue to serve as a primary marketing tool for motion picture studios and their parent conglomerates. Fan involvement with film trailers is a burgeoning area of contemporary film marketing and new media, but scholarship on this subject is lacking. Therefor, how participatory cultures connect to older film titles in online spaces through the fan trailer remains an unexplored avenue of study for cinema and media scholars.

Fig. 3: The film’s main titles are appropriately positioned in the concluding seconds of the Paramount Movie’s YouTube-exhibited trailer.

Fig. 3: The film’s main titles are appropriately positioned in the concluding seconds of the Paramount Movie’s YouTube-exhibited trailer.

Trailers in Transition: A Brief History and Contemporary Definitions

The most time-honored marketing strategy for film promotion is the movie trailer, commonly referred to as the “preview.” Kernan (2009) traced the genealogy of film trailers to 1919, citing the National Screen Service (NSS) as the first unified company responsible for creating these advertising spots. The author asserts that the evolution of the film industry during the 20th century affected changes in the types of motion pictures produced, thereby altering the aesthetics and meta-messages of trailers in the ensuing decades. A transition in film marketing occurred during the 1970s, and then again in the 1980s, with a rise in independent filmmaking, an upsurge of art-house theaters, and eventually, the summer blockbuster. During these decades, films trailers debuted on network television in thirty-second spots, visually supported by moments lifted from the film, and complete with the now-familiar and once-prominent voice-of-God narration. By the contemporary era, trailers had become “unique form[s] of narrative film exhibition, wherein promotional discourse and narrative pleasure are conjoined (whether happily or not)” (Kernan 2009b, 1). In essence, this period saw the rise of distinct promotional film advertisements alongside the audience’s familiarity and ability to detect such media forms.

Scholars regard the modern film trailer as both complex and historically shifting media type. A leading scholar on the history and transition of motion picture trailers suggests that these forms are specifically targeted, easily recognizable visual media that are created to capture, direct, and guide viewer attention (Wyatt 1994). Today, both media entities and online film fans aid in determining trailer standards and audiovisual elements. Trailers are guided by audiovisual messages through structured narratives to connect with the largest number of viewers through multi-platform distribution. Some have argued that film trailers in the digital era are defined by their dynamic if fleeting presence, asserting that contemporary trailers are forced to compete with other media forms to encourage audience-driven participation or feedback (see Rombes 2009a). Smartphones and digital tablets indicate an increase in trailer mobility and interactivity on behalf of audiences, who are receiving different media in shorter, eye-catching bursts (Grainge 2011).  Scholars have also argued that the efforts of fans on the Internet extend film capital beyond traditional home video or cable and network replay through film mashups or distributing abridged content (e.g., Sickels 2011c; Hoyt 2010a). Tyron (2009) traced the inception of the digital movie trailer to a fan preview for The Shining (Stanley Kubrick 1980) that gained Internet traction the same year as the inception of YouTube. According to the author, the fan trailer was an outgrowth of DVD culture “that allowed viewers to recognize that texts were ready to be ripped apart and reassembled in playful new ways” (151). Lazzarato (2006) described these types of fan creations as influential because they are “activities involved in defining and fixing cultural and artistic standards, fashions, tastes, consumer norms, and, more strategically, public opinion” (132). In sum, film fans use popular film properties to engage with and further promote such content to a wider range of consumers.

Re-appropriating and exhibiting film content is oftentimes understood as a group effort. Rose (2012a) argues that the cyclical discourse that occurs in online social networks encourages is what engages users to interact with film properties. Citing Avatar (James Cameron 2009) and The Lord of the Rings (Peter Jackson 2001-2003) trilogy as examples, the author maintains that a strong and relatable narrative or story is of the key to fan involvement. According to Rose, online visual narratives must be able to entertain as well as challenge participant-viewers, thereby encouraging individuals to take part in the creative act (233). Through user-participation and online media channels, the modern film trailer appears in transition. In an environment increasingly dominated by new media platforms and social networking, video-sharing websites are stimulating the development of relationships among social actors.

Defining Participatory Cultures and Digital Cinema

Participation raises the question of whose story is it? And, the answer I think is, it’s all of ours. In order to really identify with the story, in some way we have to make it our own.

-Frank Rose, The Art of Immersion (2012b).

­

Online users are now affecting many aspects of the motion picture industry and most recently, have turned to collaborative involvement with film trailers. Through an increasing number of video streaming and sharing websites, fans are producing and exhibiting short and hybrid motion picture forms from existing film content. Jenkins (1992) defined networked individuals who engage with and repurpose existing media materials as members of participatory cultures. These persons “speak from a position of collective identity, forge an alliance with a community of others in defense of tastes which…cannot be read as totally aberrant or idiosyncratic” (23). The author attributed the roots of this phenomenon to fan communities that built up around popular television programs, such as Star Trek, and who communicated and bonded through sharing information at conventions and fan clubs. More recently, Jenkins (2006a) has adapted his definition to include new media and social networking. Although optimistic about the endeavors of participatory cultures, Jenkins has noted the drawbacks of these communities as well, including the shifting power dynamics of group members and the involvement of corporate entities. In addition, the author has described the illegal activities of some members of participatory cultures, specifically those parties who undermine media conglomerates through acts of digital piracy and copyright infringement. Jenkins (2006b) has also commented on the burgeoning relationship between participatory cultures and digital cinema:

[I see] media fans as active participants…seeing their cultural products as an important aspect of the digital cinema movement. If many advocates of digital cinema have sought to democratize the means of cultural production and distribution to a broader segment of the general public then the rapid proliferation of fan-produced Star Wars films may represent a significant early success story for that movement (551-552).

In other words, the upsurge in digital cinema is dependent on fans in much the same way that fans are dependent upon interacting with cinematic creations. Digital cinema, as such, is oftentimes described as an outgrowth of online fan participation. Rombes (2009b) claims that collective acts of nostalgia, personal expression, and the adaptation of new technologies play a role in shaping digital cinema. Beginning with the rise of digital video and cinematography in the mid-1990s, the author discusses an additional factor in the relationship between digital cinema and the actions of participatory cultures: “There is a tendency in digital media – and cinema especially – to reassert imperfection, flaws, an aura of human mistakes to counterbalance the logic of perfection that pervades the digital” (Rombes 2009c, 2). In consideration with Rose’s (2012) insistence on powerful storytelling, Rombes argues that digital cinematic forms are generated and desirable because of factors such as pixilation and noise, which appear to mirror human imperfections. While fan intervention in existing film content raises questions for the future of digital cinema and a general understanding of what constitutes motion picture archives, participatory cultures have contributed to film marketing and promotion since the late 1990s. According to Erickson (2009a), who studied Internet film campaigns for The Blair Witch Project (Daniel Myrick and Eduardo Sanchez 1999) and others, studios appropriate fan-based advertising strategies if fan efforts prove financially successful.  This article is concerned with how participatory cultures repurpose and interact with the content of older motion picture titles. The entrance of fan trailers through online video streaming platforms suggests new territory for digital cinema, as well as the possible extension of the lifespan for archived film properties.

Fig. 4: A still image from the opening titles of a student-generated video essay for Days.

Fig. 4: A still image from the opening titles of a student-generated video essay for Days.

Case Study Film: Days of Heaven

Since it was first released, “Days of Heaven” has gathered legends to itself…[it] is above all one of the most beautiful films ever made. Malick’s purpose is not to tell a story of melodrama, but one of loss. His tone is elegiac. He evokes the loneliness and beauty of the limitless Texas prairie.

-Roger Ebert (1997a).

In the contemporary media marketplace, conglomerates and studios overseeing film distribution and exhibition pay close attention to the role of technologies in film promotion and branding. This is also true when considering how older film titles are released, with potential revenue gained from cable and network television broadcasts, DVD rentals and sales and most recently, VOD. Those with a financial stake in film archives oftentimes publicize and rerelease only a select number of dated film titles per year, with those properties having the most commercial potential regarded as particularly valuable on the marketplace. While some noteworthy and popular motion picture titles are available for little-to-no pay through video-sharing online services, media conglomerates use Netflix, Hulu, Amazon, and iTunes, for instance, to promote their most commercially viable films. It is here that the role of participatory culture and the evolution of the fan trailer in the archival value of film properties must be taken into consideration. Days of Heaven is significant because of its longstanding popularity amongst fans, its continual re-emergence in the public arena, and its location in cinematic history. Malick’s film arrived at a turning point in the New Hollywood of the 1970s. The competition between fledgling studio productions and a burgeoning independent film movement marked much of the decade’s releases (see Thompson and Bordwell 2010; Biskind 1998, et. al.). “But by the late 1970s,” Thomson (2012) writes, “there began to be fewer grown-up pictures meant to disturb and provoke” (459).

Before and after its release, Days of Heaven was considered an oddity for Paramount Pictures, a none-too-profitable feature that rested on the short reputation of its filmmaker.[1] Malick spent his early years in Hollywood penning several projects for other directors until his first feature-length film, Badlands (Terrence Malick 1973), gained traction from both audiences and critics, garnering a reputation as the second Bonnie and Clyde (Arthur Penn 1967). Patterson (2007) said Malick’s film offered the director the chance to “work outside more conventional parameters” (28). The filmmaker’s follow-up, however, was grander in scope and presented to audiences as a thematic American period piece. Set in the Great Plains of the 1910s, the narrative focused on a romantic amongst two migrant workers and a land baron. Morrison and Schur (2003) described Days as “wed[ding] Whitman’s poetic ideal of the democratic vista to the interior landscapes of Henry James, with a plot that evokes The Wings of the Dove and ends with a quasi-biblical plague of locusts” (23) [Fig. 1]. Indeed, the locusts were memorable, as was a lengthy scene in which wildfire spreads rapidly across the grasslands, scorching a vast swath of farmland. But much of the film’s storyline involved the happenings of Malick’s starring quartet – Richard Gere, Brooke Adams, Linda Manz, and Sam Shepard – with the characters’ muted emotions drawn out in close-ups paired with character voiceover.

Fig. 5: Gere and Adams’s characters traveling atop a railcar with other migrants in the film.

Fig. 5: Gere and Adams’s characters traveling atop a railcar with other migrants in the film.

Much of the film’s legend was only realizable years after its release. For one, Malick departed from filmmaking for two decades after Days, leaving a questionable legacy for a motion picture whose long-term stability rested on the director’s reputation and the film’s much-discussed cinematography. Over time, those perfectly composed images of man and nature, or what Kehr (2011) glowingly referred to as, “aesthetic shock effects [that] create vast, harmonious wholes,” were responsible for keeping the film in the minds of journalists and cinephiles (23-24). The film’s cinematography eventually became something of Hollywood lore [Fig. 2]. Ebert (1997b) detailed the infighting between credited director of photography, Nestor Almendros, and his predecessor, the notoriously cantankerous Haskell Wexler, in his “Great Movie” review of the film. Over the years, rumblings over credit for the look and feel of the film have led to a reconsideration of the man responsible for capturing such well-regarded images. In the years since its release, Malick returned to filmmaking and has garnered generally favorable reviews and some commercial success.[2] No fewer than ten book-length volumes are dedicated to the filmmaker’s resurgence, including The Terrence Malick Handbook (Smith 2012), and a number of academic and trade journal entries have surfaced on the canonical worthiness of Days (e.g., Crofts 2001; Woessner 2011; Koehler 2013, et. al.). Not surprisingly, praise and frustration for the film reigns on the Internet as well. The number and popularity of video clips available on video streaming and sharing websites suggests additional enforcement of the scholarly and journalistic discourse devoted to the film as well. While Days remains a much-debated and discussed film more than 35 years after its theatrical release, the role of trailers for the film on the Internet deserves attention in the era of cross-platform film promotion.

Selection of Trailer Case Studies: The Presence of Days of Heaven Online

The “Paramount Movies” channel on YouTube, overseen by Viacom, offers an original trailers for Days of Heaven [Fig. 3]. The Criterion Collection, responsible for marketing and distributing the Blu-ray and HD-DVD versions of the film, also displays an official trailer on its homepage for Days.[3] Mysteriously, Paramount’s trailer has received few visitors on YouTube while Criterion’s showcases an impressive 153 user-generated comments. The seeming randomness of attracting viewers to trailer content in online spaces is represented in this brief comparison, which appears to crossover to fan trailers as well (YouTube 2014c; The Criterion Collection 2014). The volume and popularity of fan trailers and video clips of Days showcased on YouTube overshadows this corporately controlled material in several ways as well. For one, the power of the video sharing website’s status as a social networking outlet is immediately evident. The “WorleyClarence” YouTube channel, for instance, has reposted an official version of Paramount’s trailer with an astonishing 360,000 views and 97 message board posts.[4] “JokerTreePictures,” described as an umbrella channel for three student filmmakers, has created a seven-minute video essay for Days that has gathered significant attention [Fig. 4]. Another YouTube user offers a promotional video compiled from scenes from Days matched with the music of Rod Stewart’s pop single, “Broken Arrow.” The sum of this content, which includes fan-exhibited interviews with the cast and crew as well as scenes lifted from the film, is evidence of the film’s presence on the Internet (YouTube 2014d).

For this study, three trailers were chosen as individual case studies based on the following criteria: 1) the recognizable differences in their audiovisual content, 2) the number of online views (i.e., “hits”), and 3) the number of message board posts or available online feedback. Two fan-appropriated trailers exhibited on YouTube were selected based on these requirements, as was the aforementioned trailer available through The Criterion Collection. The necessity of the trailer selection process was to compare and contrast elements of fan trailers with an official trailer approved by a media outlet in an effort to answer the research questions for this study. Many trailers that did not meet the research criteria were not selected because of factors such as conflicting content with the selected trailers, a lack of available user-generated discourse on message boards, and/or the number of recorded views or hits online. After completing the selection process, trailers were coded A (“WorleyClarence” YouTube Channel), B (“cnharrison” YouTube Channel), and C (The Criterion Collection), respectively. The researcher conducted individual and comparative audiovisual analyses on trailers A, B, and C and made notes on narrative structure and trailer content. This was followed by a qualitative content analysis of the online commentary or feedback on the message boards for each trailer’s webpage. In effect, the trailer selection process and resulting analyses were guided by the research questions for this study: What are the content-related (i.e., audiovisual) similarities and differences between the distributor-controlled, official trailer and the fan trailers under study? And what role(s) does user-generated commentary or feedback play for these trailers?

YouTube. 2008. “Days Of Heaven – Trailer (1978).” Last modified April 17, 2008.

YouTube. 2013. “Days of Heaven – Trailer.” Last modified on April 13, 2013.

YouTube. 2013. “Days of Heaven–A Video Essay.” Last modified on October 16, 2013.

Fig. 6: Adams and Shepard photographed in silhouette, with the symbolic farmhouse looming in the background.

Fig. 6: Adams and Shepard photographed in silhouette, with the symbolic farmhouse looming in the background.

Days of (Online) Fan Trailer Heaven

Trailer A opens with an image of Paramount Pictures’ trademark logo. The studio’s signature emblem fades into an image of brooding clouds looming over a wind-worn prairie. Thunder bellows on the soundtrack, and a shot of a bird of prey morphs into a backlit figure of a man standing in the grasslands at sunset. “In 1916, America was changing,” the narrator says in the trailer’s opening seconds. An image of a railcar passing over a bridge fades into a scene of factory workers digging through heaps of coal, followed by another wide frame of an empty sunbaked wheat field. The viewer is then swept into close-ups of the rough-hewn faces of the film’s stars – Gere, Shepard, and Adams – amidst passing railcars and horse-drawn carriages en route to the barren frontier [Fig. 5]. One minute and fifteen seconds into Trailer A, the serene mood and tone of the narrative changes abruptly. The narrator’s voice states that the film is “the story of a man who had nothing…the woman who loved him…and the man who would give her everything for a share of that love” (YouTube 2014e). With these words, the imagery moves away from the thematic scope of the land and its inhabitants and into the romantic dilemma at the heart of the film. A scene in which Gere’s field hand runs from law enforcement on horseback is juxtaposed with a quieter moment of his character embracing Adams in a quiet meadow. The next shot is an extreme close-up of Shepard’s watchful gaze, as if overseeing these scenes from afar.

As the narrative for Trailer A moves towards its conclusion, Adams and Shepard are photographed in silhouette inside the latter’s large estate, while the bedraggled face of Gere’s character peers up at the duo through a windowpane from below. This moment is framed from Gere’s perspective, with the actor and the encompassing field bathed in the deep blues of a Midwestern dusk, suggesting the loneliness his character will face with the coming of night. The film’s title appears over this closing shot, foreshadowing a troubled outcome for the trio. Trailer A presents much of the entire film’s narrative in under two minutes; what begins as a broad glimpse of turn-of-the-century westward expansion in the U.S. evolves into a minor tale of lost love [Fig. 5]. Thematically, the trailer’s primary audiovisual message suggests a heightening of nostalgia for both the American West and the Hollywood of the late 1970s, with the mythic qualities of innocence and utopia highlighted in the cinematography and production design [Fig. 6]. The professionalism of the editing in Trailer A, including the pairing of shots and sequence evolution provides a seamless story arc. Thus, the inclusion of Paramount’s introductory logo, the ‘70s-era voice-of-God narration, and the production elements suggests that this user-exhibited fan trailer was re-appropriated without revising the original trailer’s content. Therefore, Trailer A is most likely an original trailer for the film repurposed by one or more online fans. Trailer B also provides a visually compelling narrative to signal nostalgia and romanticism for the American West. But here, the viewer is immediately transplanted into to the lives of the film’s primary characters without the broader introduction of the land and its inhabitants as witnessed in Trailer A [Fig. 7].

Fig. 7: The film’s use of natural light to emphasize dramatic elements is also highlighted within the trailers.

Fig. 7: The film’s use of natural light to emphasize dramatic elements is also highlighted within the trailers.

The opening shot in Trailer B, a striking low-angle image of Gere, Adams, and the younger Manz running to catch a moving train, introduces the film’s predominant family dynamic.[5] Next is a shot of moving railcars topped with migrant travelers that segue into multiple close-ups of these characters’ hardened faces. Already, the viewer is guided toward the themes of travel and migration. The following image shows the Gere, Adams, and Manz trio atop one of the railcars, amidst the masses, fleeing the East for better opportunities. The rest of Trailer B’s running time focuses on the romantic triangle that ensues. Several important elements in Trailer B suggest a greater degree of user- repurposing. Manz’s tinny backwoods drawl, taken from the film’s narration, guides the trailer’s audio track for much of the running time, and is backed by a second musical track of delicately plucked guitar strings. In addition, the caption for Trailer B, located just below the video player on YouTube, states, “Bill, Abby, and sis arrive on the panhandle,” a sentiment only marginally correlated with the majority of the trailer’s visual narrative (YouTube 2014f). Another item that speaks to user re-appropriation is the individual shot duration, which moves at a more leisurely pace here, and seems to have been edited mostly to match Manz’s voiceover.

Further suggestive of fan involvement with Trailer B’s content is the abrupt segue from Manz’s voice and the guitar string audio tracks to the ambient sounds of trotting horses and rolling wagon wheels. Visually, the nonprofessional editing is emphasized at this point as well, with a sequence in which Gere’s character is propositioned for work by a land baron, a moment that is abruptly interrupted by a long shot of migrants moving en mass across the prairie. Throughout the two and a half-minute running time for Trailer B, the mood and tone shift in favor of different scenes from the film that drive the trailer towards a questionable conclusion. Marketing and film promotion is immediately evident on the webpage for Trailer C [Fig. 8]. The Criterion Collection offers viewers the option of purchasing several DVD versions of the film, reading a written essay on the film’s historical significance, a list of DVD special features, and links to related films from the company in addition to the trailer.

The trailer itself, however, is constructed from film content not included in Trailers A and B. In this much-abridged version, the guitar audio track preceding Manz’s narration is audibly fragmented and disassociated from any cohesive visible narrative. As such, the film’s primary visual content is made up of close-ups of the nondescript faces of migrants overlooking a land of grazing crows and antelope on the abandoned prairie. Here, Manz’s brief narration serves to introduce the film’s quiet mood and leisurely pacing. The aforementioned scene of Gere interacting with the land baron is cut prematurely in Trailer C, presumably for purposes of keeping the trailer’s length under the running time of one minute. In this version, the scene that introduces the bullhorn-gripping farm owner is interrupted by an establishing crane shot that places the viewer in the midst of migrants scampering towards the opportunity of work. Each of these moments take up several seconds worth of running time, and Criterion’s trailer closes abruptly with a surprising fade-to-black.

Fig. 11: Criterion’s webpage for Days of Heaven offers visitors a number of options to interact with the film.

Fig. 11: Criterion’s webpage for Days of Heaven offers visitors a number of options to interact with the film.

Whereas the finales of both fan-appropriated trailers on YouTube are classically structured to mirror the resolutions found in many trailers of the 1970s, the transition to a black frame in Trailer C suggests a different kind of closure. The trailer concludes by returning to a still frame of six farmhands standing in awe of an insect downpour, a somewhat iconic image from the famous “locust scene” in the film. This visual placeholder is representative of Criterion’s idyllic version of the film’s significance. As such, this striking still image speaks directly to curating the memory of Days, arguably more so than the totality of the narrative for Trailer C. Although the design of the distributor’s webpage is simultaneously content-heavy and visually arresting, this emblematic still frame stands apart, begging the visitor to click, watch or re-watch and possibly, purchase the film from the distributor.

Feedback on Heaven: The Online Discourse of Cinematic Aesthetics & Nostalgia

The contents of three hundred user-generated message board posts for Trailers A, B, and C were analyzed for this study. Most of this feedback was found to be praiseworthy of Days, with many of the user-posts lauding the film’s cinematography. The discourse on Criterion’s webpage for the film was overwhelmingly positive and found to reflect the distributor’s marketing intentions. “A beautiful spectral and view of the early 1900s mid-western America,” Mike Santoro wrote on the message board. “I love Malick’s brilliant direction in this [film]” (The Criterion Collection 2014b). Others commentators on this webpage used specific discourse that intertwined aspects of their real-world lives with the film’s history and nostalgia. “My first Malick movie, discovered when I was watching every movie on rogerebert.com’s ‘101 Movies To See Before You Die,’” Taylor P. stated. Bennett Duckworth wrote, “…thanks Dad for introducing this movie to me.” And mimicking Manz’s drawl in the character’s narration, Arthur Mhoyan said, “There were people sufferin’ in pain and hunger. Some people their tongues were hangin’ out of their mouths” (The Criterion Collection 2014c).

While single-word and somewhat elusive statements, such as “Breathtaking” and “Beautiful,” were found on the Criterion message board as well, much of the feedback was more detailed and descriptive. The lack of negative comments on the message board is further indicative of Criterion’s approach to online publicity and distribution for the film. In turn, the majority of user-feedback for Trailers A and B on YouTube was specifically targeted at the film’s cinematography. Equal parts excitement and praise for the film’s imagery was evident on both message boards, suggesting that the film’s visual approach is endorsed through fan-recall on these video-streaming webpages. For example, the “GregF” channel wrote, “…all 5 [of] Malick’s movies are beautiful but there are no words to describe Days Of Heaven…pure magic.” The “44eelz” channel posted, “i haven’t seen this movie yet but the cinematography looks amazing.” The “ErikHutt” channel added that “[Days] was shot in Alberta,” and the “MrKeepitunderyourhat” channel said, “To be honest, I’d say that the most famous aspect of the entire film is its magic hour cinematography” (YouTube 2014g).

The similarities in the content and tone of the statements analyzed across all three webpages suggest that fans are fond of the film’s historical significance and imagery. The cause-effect nature of this discourse also acts as an effort to keep the film in memory while promoting it to others. The content of this rhetoric also signifies the film’s ability to evoke an era in Hollywood history in which aesthetic power swayed and captivated audience members. In sum, much of this online discourse speaks to how film fans in online spaces curate the myth and nostalgia of aging mainstream film properties. Much of these statements reflect a sincere familiarity with Malick’s production design and the aesthetic properties of the cinematography. The statements under analysis, therefore, speak to the role of message boards in film advertising as well as the intricacies of fan-generated promotional feedback.

Promoting Hollywood Through the Fan Trailers: The Archive in Transit

YouTube. 2015. “Honest Trailers.” Accessed February 11, 2015.

This article investigated how participatory cultures use fan trailers to engage with aging Hollywood titles in online spaces. The findings suggest that online film fans utilize fan trailers to interact with others while drawing attention to archival film properties. In effect, the findings from this study demonstrate several ways in which trailer repurposing and exhibition on the Internet aids in developing fan support around older motion pictures. An upsurge in fan trailers on the Internet is a burgeoning avenue of marketing for Hollywood studios and film distributors. Through new media platforms, fan trailers have the potential to reach global audiences and encourage social networking and commentary. In this study, the number of fan trailer views and user-generated message board posts was found to play a role in supporting interest in online film content. The audiovisual elements of both fan trailers for this study were generated from existing film content and repurposed to varying degrees. Specifically, the fan-edited trailer content was found to draw attention to the emotive properties of the film text. Collectively, the trailer narratives for this study presented an overwhelmingly favorable image of the case study film, as well as its historical significance and nostalgic qualities. The textual or written discourse analyzed in message boards on the webpages under investigation was found to shape the collective memory of the case study film as well. The content from this portion of the analysis also helped in preserving a positive view of the film itself, with much of the user-generated feedback positioned to promote the film’s cinematography and production design.

The composite findings indicate that fan trailers play a detrimental role in reviving older studio properties. The unintended consequences of these actions suggest a new avenue for media conglomerates and/or film distributors in marketing older motion pictures in the digital era. With Hollywood making fewer “midrange films [with] distinctly American subject matter,” such as Days of Heaven, smaller production companies and independent channels are overtaking this once-profitable market (Goldstein 2012). The role(s) taken on by members of participatory cultures, as well as the long-term effects of their interventions in online spaces, remains to be seen. For aging Hollywood film, fan trailers appear to offer one example of a promotional tool for film distribution and archiving. In June 2015, more than 88 million viewers had accessed 107 mock fan trailers through Honest Trailers, the YouTube-hosted channel by Screen Junkies (YouTube 2015). As Erickson (2009b) suggested, “with rapidly evolving technological features and equipment, tomorrow may yield an entirely new approach to using the Internet in a film promotion campaign” (51). As technological advancements in cinema and digital media continue to unfold, new online platforms and Web channels are creating an increasing number of spaces for participatory cultures and motion pictures. While many of these changes are on the horizon, scholars have predicted a continuous stream of content-related interruptions from tech-savvy film fans, as well as an evolution in the blending of virtual selves with cinematic information in cyberspace (e.g., Hansen 2006; Hardt and Negri 2004). Although the art of re-appropriating film content on the Internet has ballooned into a truly mass phenomenon, the future and direction of the fan trailer will depend on the negotiated balance between online cinephiles and digital control of motion picture properties.

REFERENCES 

Bettig, Ronald V. 2008. “Hollywood and intellectual property.” In The Contemporary Hollywood Film Industry, edited by Paul McDonald and Janet Wasko, 195-205. Malden, MA: Blackwell, 2008.

Biskind, Peter. 1998. Easy Riders, Raging Bulls: How the Sex-Drugs-and-Rock-‘n’-Roll Generation Saved Hollywood. New York: Simon & Schuster.

Boyle, James. 2008. The Pubic Domain: Enclosing the Commons of the Mind. New Haven, CT: Yale University Press.

Cunningham, Stuart and Jon Silver. 2013. Screen Distribution and the New King Kongs of the Online World. New York, NY: Palgrave Macmillan.

Crofts, Charlotte. 2001. “From the ‘hegemony of the eye’ to the ‘hierarchy of Perception’: The reconfiguration of sound and image in Terrence Malick’s Days of Heaven.” The Journal of Media Practice 2, no. 1: 19-29.

Ebert, Roger. 1997. “Great Movie: Days of Heaven.” RogerEbert.com., December 7. Accessed January 20, 2015.

Erickson, Mary P. 2009a. “Co-opting ‘independence’: Hollywood’s marketing label.” In The Business of Entertainment: Movies, Vol. 1, edited by Robert Sickels, 129-152. Westport, CT: Praeger Publishers.

Erickson, Mary P. 2009b. “KingKong.com verses LOLTheMovie.com: Toward a framework of corporate and independent online film promotion.” In The Business of Entertainment: Movies, Vol. 1, edited by Robert Sickels, 37-54. Westport, CT: Praeger Publishers.

Franich, Dennis. 2013. “Do audiences actually WANT shorter movie trailers?” Entertainment Weekly, June 7. Accessed November 13, 2014.

Goldstein, Patrick. 2012. “Hollywood’s global strategy: Made in America, but not for Americans.” 24 Frames: Movies: Past, Present and Future (Los Angeles Times blog), January 10. Accessed December 17, 2014.

Grainge, Paul. 2011. Ephemeral Media: Transitory Screen Culture from Television to YouTube. New York: Palgrave Macmillan.

Hansen, Mark B. N. 2006. Bodies in Code: Interfaces with Digital Media. New York: Routledge.

Hardt, Michael and Antonio Negri. 2005. Multitude: War and Democracy in the Age of Empire. New York: Penguin Books.

Hoyt, Eric. 2010. “The future of selling the past: Studio libraries in the 21st century.” Jump Cut: A Review of Contemporary Media 52. Accessed January 18, 2015.

Internet Movie Database. 2015. “Days of Heaven.” Accessed March 12.

Jenkins, Henry. 1992. Textual Poachers: Television Fans & Participatory Culture. New York: Routledge, Chapman and Hall, Inc.

Jenkins, Henry. 2006a. Convergence Culture: Where Old and New Media Collide. New York: New York University Press.

Jenkins, Henry. 2006a. “Quentin Tarantino’s Star Wars?: Digital cinema, media convergence, and participatory culture.” In Media and Cultural Studies: Keyworks, edited by Gigi Durham Meenakshi and Douglas Kellner. Malden, MA: Blackwell Publishing Ltd.

Johnston, K.M. 2008. “’The coolest way to watch movie trailers in the world’: Trailers in the digital age.” Convergence 14, no. 2: 145-160.

Kehr, Dave. 2011. “Days of Heaven.” In When Movies Mattered: Reviews from a Transformative Decade, 23-27. Chicago, IL: University of Chicago Press.

Kernan, Lisa. 2009. Coming Attractions: Reading American Movie Trailers. Austin, TX: University of Texas Press.

Koehler, Robert. 2013. “What the hell happened with Terrence Malick?” Cineaste 38, no. 4: 4-9.

Lazzarato, Maurizio. 2006. “Immaterial labor.” In Radical Thought in Italy: A Potential Politics, edited by Paolo Virno and Michael Hardt, 132-146. Minneapolis, MN: University of Minnesota Press.

Litman, Jessica. 2002. “War stories.” Cardozo Arts & Entertainment Law Journal 20, no. 2: 337-365.

Morrison, James and Thomas Schur. 2003. The Films of Terrence Malick. Westport, CT: Praeger Publishers.

Obst, Lynda. 2013. Sleepless in Hollywood: Tales from the New Abnormal in the Movie Business. New York: Simon & Schuster.

Patterson, Hannah. 2007. “Two characters in search of a direction: Motivation and the construction of identity in Badlands.” In The Cinema of Terrence Malick: Poetic Visions of America, edited by Hannah Patterson, 27-39. London: Wallflower Press.

Perren, Alisa. 2010. “Business as unusual: Conglomerate-sized challenges for film and television in the digital arena.” Journal of Popular Film and Television 38, no. 2: 72

Rombes, Nicholas. 2009. Cinema in the Digital Age. New York: Wallflower Press.

Rose, Frank. 2012. The Art of Immersion: How the Digital Generation Is Remaking Hollywood, Madison Avenue, and the Way We Tell Stories. New York: W.W. Norton & Company.

Rothman, Lily. 2014. “Movie trailers will get shorter, but won’t become interactive anytime soon.” TIME, January 27. Accessed October 3, 2014.

Roxborough, Scott. 2013. “Global on-demand revenues to top $6 billion by 2018.” The Hollywood Reporter, August 14. Accessed September 8, 2014.

Sickels, Robert. 2011. American Film in the Digital Age. Santa Barbara, CA: Praeger Publishers.

Smith, Emily. 2012. The Terrence Malick Handbook – Everything You Need to Know about Terrence Malick. Unknown Location: Tebbo.

Sterbenz, Christina. 2014. “How sketchy streaming sites really work – and why some are illegal.” Business Insider, April 24. Accessed January 10, 2015.

The Criterion Collection. 2015. “Days of Heaven: Terrence Malick.” Accessed on October 22, 2014.

Thompson, Kristin and David Bordwell. 2010. Film History: An Introduction (3rd ed). New York: McGraw-Hill Higher Education.

Thomson, David. 2012. The Big Screen: The Story of the Movies. New York, NY: Farrar, Straus and Giroux.

Tolson, Andrew. 2010. “A new authenticity? Communicative practices on YouTube.” Critical Discourse Studies 7, no. 4: 277-289.

Tyron, Chuck. 2009. Reinventing Cinema: Movies in the Age of Media Convergence. New Brunswick, NJ: Rutgers University Press.

Woessner, Martin. 2011. “What is Heideggerian cinema? Film, philosophy, and cultural mobility.” New German Critique 38,  no. 2: 129-157.

Wyatt, Justin. 1994. High Concept: Movies and Marketing in Hollywood. Austin, TX: University of Texas Press.

 

Filmography

Avatar. 2009. Directed by James Cameron. USA: 20th Century Fox.

Bonnie and Clyde. 1967. Directed by Arthur Penn. USA: Warner Brothers.

Days of Heaven. 1978. Directed by Terrence Malick. USA: Paramount Pictures. 

Gone Girl. 2014. Directed by David Fincher. USA: 20th Century Fox.

The Blair Witch Project
. 1999. Directed by Daniel Myrick and Eduardo Sanchez. USA: Haxan Films.

The Lord of the Rings: The Fellowship of the Ring. 2001. Directed by Peter Jackson. USA: New Line Cinema.

The Lord of the Rings: The Two Towers. 2002. Directed by Peter Jackson. USA: New Line Cinema.

The Lord of the Rings: The Return of the King. 2003. Directed by Peter Jackson. USA: New Line Cinema.

YouTube. 2008. “Days Of Heaven – Trailer (1978).” Last modified April 17, 2008.

YouTube. 2009. “Days of Heaven – Terrence Malick (1978).” Last modified on November 9, 2009.

YouTube. 2013. “Days of Heaven – Trailer.” Last modified on April 13, 2013.

YouTube. 2013. “Days of Heaven–A Video Essay.” Last modified on October 16, 2013.

YouTube. 2015. “Honest Trailers.” Accessed February 11, 2015.


Notes
[1]
Days of Heaven’s 1978 box-office gross was $3.5 million nationwide. Compare this figure to other mainstream studio releases of 1978 that received Oscar attention and critical acclaim, such as Heaven Can Wait ($81.6 million) (Beatty 1978), The Deer Hunter (roughly $49 million) (Cimino 1978), and Midnight Express ($35 million) (Parker 1978) (BoxOfficeMojo 2014).

[2] At the time of this writing, three Malick-directed films are in various stages of development, with his next feature, Knight of Cups, scheduled for wide release in 2015.

[3] The one-hour, thirty-three minute feature film is also available for rent or purchase on YouTube.

[4] Paramount Pictures’ YouTube channel displays fewer than 4,000 posts.

[5] This image is also used near the end of Trailer A, primarily to symbolize the passage of time for migrants moving from urban to rural areas.

Bio:

Kyle R. McDaniel is a doctoral candidate in the School of Journalism and Communication at the University of Oregon. His research interests include the intersections between American cinema and digital culture in the 21st century. His forthcoming dissertation focuses on the usage and repetition of visual effects in contemporary documentary film.

 

Volume 26

Contents

  1. “Children should play with dead things”: transforming Frankenstein in Tim Burton’s Frankenweenie –  Erin Hawley
  2. “You gave me no choice”: A queer reading of Mordred’s journey to villainy and struggle for identity in BBC’s Merlin  –  Joseph Brennan
  3. Days of YouTube-ing Days of Heaven: Participatory Culture and the Fan Trailer  –  Kyle R. McDaniel
  4. When a Good Girl Goes to War: Claire Adams Mackinnon and Her Service During World War IHeather L. Robinson 
  5. ‘Rock‘n’roll’s evil doll’: the Female Popular Music Genre of Barbie Rock  –  Rock Chugg
  6. Morality, Mortality and Materialism: an Art Historian Watches Mad Men – Catherine Wilkins
  7. Playing At Work  –  Samuel Tobin
  8. 1970s Disaster Films: The Star In Jeopardy Nathan Smith

 

 

“Children should play with dead things”: transforming Frankenstein in Tim Burton’s Frankenweenie – Erin Hawley

Abstract: In this paper, I explore the possibility of retelling Mary Shelley’s novel Frankenstein in a children’s media text.  Like most material within the horror genre, Frankenstein is not immediately accessible to children and its key themes and tropes have traditionally been read as articulations of “adult” concerns.  Yet Frankenstein is also a tale with surprisingly child-centric themes.  With this in mind, I consider how the Frankenstein tale has been transformed within the constructed space of a child’s worldview in Tim Burton’s 2012 animated film Frankenweenie.  I argue that the film neither simplifies nor expresses great fidelity to Shelley’s novel, but instead cultivates a sense of curiosity and cultural literacy regarding the Frankenstein tale and the horror genre itself.

Sparky the dog. Frankenweenie (Tim Burton, 2012)

Sparky the dog. Frankenweenie (Tim Burton, 2012)

The horror genre has long been considered “off limits” to children.  From the rewriting of fairytales to erase their violent and scary content (Zipes 1993) to the literal defacement of eighteenth century children’s literature to remove traces of the Gothic (Townshend 2008), efforts to disentangle children’s texts from horror have given rise to the notion that children cannot derive the same sort of pleasure from “being scared” that adults can.  Recent scholarship has suggested, however, that children can and do take pleasure in horror material.  In her work on child cinema audiences in Britain, Sarah Smith has found that horror films in the 1930s were “extremely popular with children” due to the “mixed feelings of fear and fun” they evoked (2005, 58).  Writing of James Whale’s film Frankenstein (1931), Smith observes that children were “fascinated by its appeal and attended in droves” (2005, 70).  Similarly, David Buckingham’s research into children as horror viewers reveals that, while fright reactions to horror material can be powerful and long-lived, child audiences also take pleasure in the conventions of the horror text – they enjoy watching “evil destroyed” but also watching it “triumph”; they enjoy the feeling of fear itself and, like adult viewers, find pleasure in horror’s momentary destabilisation of societal norms (1996, 112-116).

The pleasures of horror from a child’s perspective have also been explored by Neil Gaiman (2006), who tells an interesting story about his daughter’s fascination with James Whale’s The Bride of Frankenstein (1935).  “My daughter Maddy loves the idea of The Bride of Frankenstein,” he writes: “she’s ten”.  Such fascination leads to dress-ups and play, and eventually to young Maddy and her friend watching the horror classic under Gaiman’s supervision.  When confronted with the movie itself, however, the enthusiasm wanes: the kids don’t get it.  As Gaiman observes, “They enjoyed it, wriggling and squealing in all the right places. But once it was done, the girls had an identical reaction. ‘Is it over?’ asked one. ‘That was weird,’ said the other, flatly. They were as unsatisfied as an audience could be”.

To some extent, this reaction is not surprising.  The Bride of Frankenstein is based on Mary Shelley’s Gothic novel Frankenstein, a text that – like most material within the horror genre – is usually read as an articulation of decidedly adult concerns.  From the original novel to its more recent manifestations in popular media, the Frankenstein tale is peppered with depictions of violence and violation, murder and misogyny; across the long history of its remaking in popular culture it has been interpreted as a story about genetic manipulation (Waldby 2002, 29), sexual transgression (Mellor 2003, 12-13), and post-partum depression (Johnson 1982, 6), to name just a few of its more adult-centric resonances.

Yet Frankenstein is also a tale with surprisingly child-centric themes.  At its heart, it is a story about what it means to be an outsider and what it means to encounter, experience, and negotiate otherness; these are themes that have more recently been explored by writers of children’s and young adult fiction from Roald Dahl to Stephenie Meyer.  As Barbara Johnson has pointed out, Frankenstein is also essentially a story about parent/child relationships: with its themes of monstrosity and technology, Johnson tells us, Shelley’s novel explores “the love-hate relation we have toward our children” (1982, 6).  Building on Johnson we can suggest that by offering us a glimpse of the world through the monster’s eyes the novel also briefly presents this “love-hate relation” from the child’s perspective, and that decades of Frankenstein movies continue this by offering the misunderstood monster as an icon of all that is unruly, confused, and frightening about childhood itself.

The story Gaiman tells about his daughter’s fascination with The Bride of Frankenstein and her reaction – “that was weird” – to the movie itself is a lovely articulation of the way children may be simultaneously drawn to and locked out of the Frankenstein tale.  It is interesting to note that Gaiman’s daughter and her friend were not frightened by the film or put off by its horror elements (indeed, they seemed to enjoy this aspect of the movie, “wriggling and squealing in all the right places”); instead, it was a certain indefinable strangeness that informed their ultimately “unsatisfied” reaction.  All this suggests that children can engage meaningfully and pleasurably with material in the horror genre, especially if that material is rewritten with a child’s perspective in mind.

In this article, I explore the relationship between Frankenstein and young audiences and consider the possibility of retelling Shelley’s novel in a children’s media text.  My analysis is inspired by the recent appearance of characters from Shelley’s novel and its various adaptations in three children’s animated films: Frankenweenie (Tim Burton, 2012), in which a boy named Victor Frankenstein reanimates his dog Sparky after a tragic car accident; Igor (Anthony Leondis, 2008), in which a hunch-backed laboratory assistant brings a female monster to life; and Hotel Transylvania (Genndy Tartakovsky, 2012), in which the Frankenstein monster and his Bride join Dracula and a host of other characters from the horror genre.  This trend towards engaging the Frankenstein myth in children’s media begs the question: how have such texts made Shelley’s tale accessible to young audiences, and with what degree of success?

Below, I take up this question with specific reference to Tim Burton’s Frankenweenie.  Not only is Burton’s film (as we shall see) the most highly regarded and in some senses the most successful of these three texts, it is also the most complex and arguably does not “dumb down” its source material.  My analysis of Frankenweenie will examine how the film constructs a “child’s eye view” and transforms the Frankenstein tale so that its characters, themes, and narratives make sense within the imagined space of a child’s world.  I will demonstrate that Burton’s film captures the spirit of its source text without necessarily striving for fidelity.  I will also consider some of Frankenweenie’s extra-textual material, exploring how reviews, product tie-ins, and even the film’s intertextual references contribute to its overall project of transforming but not simplifying the Frankenstein tale for children.

Adaptation, simplification, and transformation

Victor and his dog Sparky from Tim Burton's homage to the Frankenstein story, Frankenweenie (2012).

Victor and his dog Sparky from Tim Burton’s homage to the Frankenstein story, Frankenweenie (2012).

Frankenweenie is a stop-motion animation inspired by Burton’s earlier live-action film of the same name.  Here, the Frankenstein tale is relocated to one of Burton’s characteristic suburbia-scapes (“New Holland”), complete with manicured lawns, hedge sculptures, and monstrously mediocre residents.  Within this new narrative space, “Victor Frankenstein” is a child: a troubled, creative loner who spends his time tinkering in the attic, playing with his beloved dog Sparky, and making movies.  Tragedy enters Victor’s life when Sparky is killed in a car accident.  Inspired by his science teacher, the delightfully dour Mr Rzykruski, Victor steals Sparky’s body from the pet cemetery, drags the corpse back to the family home, and reanimates him in the attic.  When Victor’s classmates learn his secret, they try to replicate the experiment.  Chaos ensues as pets both living and dead are transformed into monsters who descend on New Holland, leading to a climactic showdown at the windmill overlooking the town.

The relationship between Frankenweenie and its source text, Mary Shelley’s Frankenstein, is complex.  Burton’s film both diverges from and intersects with Shelley’s novel, defining itself through patterns of fleeting fidelity and moments of spectacular transformation.  At the same time, the film makes reference to a plethora of other texts both within and beyond the Frankenstein mythos, thereby demonstrating the ways in which “adaptation” approaches and merges with “intertextuality” (see Elliott 2014; Martin 2009; Leitch 2003).  In other words, Frankenweenie is by no means a “faithful” retelling of Shelley’s Frankenstein.  It should be noted, however, that Shelley’s novel – despite being adapted many times in the centuries since its publication, across different media and in different genres – has not tended to inspire fierce fidelity in adapting authors.  As Albert Lavalley points out, Frankenstein tends to be “viewed by the playwright or the screenwriter as a mythic text, an occasion for the writer to let loose his own fantasies or to stage what he feels is dramatically effective, to remain true to the central core of the myth, [but] often to let it interact with fears and tensions of the current time” (1979, 245).

The notion of “fidelity” to an original text as the means of measuring an adaptation’s success, strength, and value has itself been thoroughly contested and problematised in recent years.  Fuelled particularly by the work of adaptation theorists such as Robert Stam (2005), Thomas Leitch (2003, 2007), and Imelda Whelehan (1999), this problematisation of the fidelity model has been an intervention in established ways of thinking about the relationship between an adaptive text and its source material.  As Will Brooker observes, though, fidelity criticism may be “outmoded and discredited within academia” but it has managed to “retain its currency within popular discourse” (2012, 45); in particular, it still informs the critical reception of films that adapt well-known novels or works of literature.  Even within academia, moreover, fidelity criticism has tended to linger in discussions of children’s media texts, particularly when the texts in question are retellings of classic or literary works.  It is often assumed that such adaptations carry some degree of responsibility for encouraging children to read and connect with the source material (Napolitano 2009, 81); in this way, the issue of fidelity becomes more urgent in the context of children’s media.

Concerns about fidelity in children’s adaptations are compounded by the issue of simplification.  Frankenweenie, for instance, is both an adaptation of a literary text and a reworking of classic horror films within the space of a child’s animation: the question we may immediately wish to ask, then, is “what has been lost in this process?”.  Both Shelley’s novel and the films of James Whale are today held in high regard as cultural classics, while the Frankenstein myth itself is a repository of ideas and cultural conversations about selfhood, embodiment, subjectivity, life, and death.  Potentially, the simplification of this myth for children would involve more than just a strategic removal of violent and sexual content in order to achieve a PG rating: it would be a process of dumbing down, a cleaning up of a story that works best when it is not “clean”.  It would also be a form of commercialisation, a reduction of a complex tale so that it can be packaged and marketed to young audiences.

These problems of simplification, commodification, and the dumbing down of source material are frequently mentioned by analysts of children’s adaptations, especially when the adaptation in question is a Disney product (as is Frankenweenie).  Writing in 1965 for the journal Horn Book, Frances Sayers refers to the “sweet” and “saccharine” nature of Disney adaptations and argues that, in order to both address and construct a child or family audience, Disney texts present life as lacking “any conflict except the obvious conflict of violence” (609).  Her concerns have been echoed by Hastings, who writes of the “conscious effort [by Disney] to produce children’s movies with no alarming moral ambiguities” (1993, 84).  Zipes, in turn, laments the way Disney has “‘violated’ the literary genre of the fairytale and packaged his versions in his name through the merchandising of books, toys, clothing, and records” (1995, 38).  Marc Napolitano’s work on the “Disneyfication of Dickens” is particularly relevant here because it explores the intersection between adult literature and children’s media.  Napolitano argues that the films Oliver & Company and The Muppet Christmas Carol – both Disney texts that retell canonical works by Charles Dickens – are “simplified and sanitized adaptations of Dickens that were marketed to families by the Walt Disney Company” (2009, 80).  In Oliver & Company in particular, Napolitano argues, Disney “lightens the material significantly and uses cute, cuddly animal characters, all of whom would be reproduced as stuffed toys, McDonald’s Happy Meal prizes… and countless other types of child-friendly merchandise to market the film to kids” (2009, 82).

All such criticism of Disney’s treatment of literary material is important, and functions as part of a wider interrogation of the seemingly “apolitical” and “critically untouchable” world of children’s animated film (Bell, Haas, and Sells 1995, 2).  As analysts of Disney products, it is essential that we disentangle ourselves from our own enjoyment of the Disney “magic”; this is part of what Zipes has called “Breaking the Disney Spell” (1995).  At the same time, however, claims about “Disneyfication” can be problematic when they make sweeping assumptions about young audiences, their levels of media literacy, and the ways in which they engage with media texts.  In other words, when accusing a children’s text of simplification we ourselves risk making an overly simplified reading of the child audience.  The charge of simplification becomes especially problematic when it lapses into what Semenza (2008) has termed the “dumbing down cliché”: the notion that adapting a literary text for children must always and automatically involve a process of reduction and commodification.

In the context of these concerns, Frankenweenie provides us with an interesting example because it resists the simplification process and simultaneously encourages its young audience to reconnect with the source material through means other than fidelity.  The film’s refusal to “Disneyfy” the Frankenstein tale is signified by the transformation of the Disney logo in the opening sequence: lightning strikes the familiar Disney castle and the picture turns black and white, a suggestion that there will be no fairies or cute, singing animals in the film that follows; that there will be no attempts to render the Frankenstein tale “safe” and “simple” even as it is opened up for young viewers.  While this transformation of the Disney logo is indicative of the mediating presence of Burton in the adaptation process, and of the supposed clash between the Disney and Burton brands, it can also be read as a resistance to simplification – a suggestion that the film will be “Frankenstein for kids” but not “Frankenstein lite”.

In what follows, I explore how Frankenweenie transforms (rather than simplifies) the Frankenstein tale within the imagined space of a child’s world.  I use the term “transformation” with an awareness of its applicability to studies of animation as an art form, a technology, and a mode of representation.  As both Susan Napier (2000) and Paul Wells (1998) have noted, animation has metamorphic qualities that distinguish it from live-action cinema and that manifest at the levels of story, body, and space.  Wells has also argued that the process of adapting a literary text into animated film can involve “an act of literal transformation which carries with it mythic and metaphoric possibility” (2007, 201).  In this way, the idea of transformation allows us to discuss children’s animated films as adaptations without making assumptions about animation as a medium (for instance, that it is inferior to live-action cinema) or about children as audiences (for instance, that they are incapable of understanding textual and intertextual complexities).

Transformation and the child’s eye view

In her analysis of the filmic adaptation of Maurice Sendak’s beloved picture book Where the Wild Things Are, Sarah Annunziato (2014) explores the construction of a “child’s eye view”, arguing that the film – while drawing attention and public comment for its scariness and mature themes – is appropriate for young audiences because it imagines the world as seen through a child’s eyes.  Similar claims can be made of Frankenweenie, which constructs a child’s view of the world and repeatedly invites its viewers to inhabit this childlike space.  In both these films, the creation of a child’s eye view specifically involves moments of scariness rather than excluding them.  The relationship between Frankenstein and Frankenweenie differs, however, from that between the book and film versions of Where the Wild Things Are because it involves a shift from adult to child audience.  In Burton’s film, therefore, employing a child’s perspective allows for a significant rethinking of the original tale.

In its simplest sense, this child’s eye view is visible in the depiction of New Holland and its residents.  While certainly reminiscent of some of Burton’s other visions of suburbia, New Holland is best described as a small town landscape seen through a child’s eyes: a place of long shadows and neat lines, of fantasy and darkness, of strange children and menacing adults.  The frequent use of low camera angles to depict some of these adult characters (such as the mayor Mr Bergermeister and the science teacher Mr Rzykruski) aligns us with Victor and invites us to adopt a child’s perspective.  While not menacing or imposing, Victor’s parents, too, are adults as seen by children: simplistic to the point of caricature, caught up in trivial or meaningless “grown-up” concerns (Victor’s father talks endlessly about his work as a travel agent; Victor’s mother is repeatedly seen vacuuming the house and/or reading romance novels).  On the other hand, the world of Victor (the child’s world) is depicted as complex, detailed, and intricate.  This is best represented by the attic, a cluttered space of creativity, invention, and play – and a notable contrast with the rest of Victor’s house and suburb, which are neat, sparse, and boring.

This inherent difference between adults and children – and the resultant conflict, always seen from the child’s perspective – is central to the plot of Frankenweenie.  From the opening scenes we learn that Victor is misunderstood by his mostly well-meaning parents, who worry that he spends too much time alone and will “turn out weird”.  His father encourages Victor to take up baseball, which leads inadvertently to Sparky’s death: the little dog meets his doom while chasing a ball hit by Victor.  The subsequent depiction of Victor’s grief is highly moving, all the more so because his parents do not seem to understand the extent of his sadness.  His mother offers clichés and platitudes: “If we could bring him back, we would” and “when you lose someone they never really leave you – they just move into a special place in your heart”, which Victor interprets as hollow and macabre (“I don’t want him in my heart”, he objects, “I want him here with me”).  These early scenes suggest a sense of turmoil beneath the calm surface of even the most loving parent-child relationship: a version, perhaps, of the “love-hate relation” that Johnson (1982, 6) detects within Shelley’s visions of monstrosity.  They also reveal that the world seen through a child’s eyes is not a simple place, even though it may be dominated by fantasies (such as the desire to bring Sparky back to life, which Victor soon fulfills).

It is through these early depictions of conflict, death, and grief that the film captures the thematic spirit of Mary Shelley’s novel.  In Shelley’s text, Victor Frankenstein is driven to create his monster by a desire to suspend mortality and escape the horrors of death and decay: Shelley’s Victor is both haunted and inspired by the death of his mother, Caroline, which leads him to seek out scientific means of “renew[ing] life where death had apparently devoted the body to corruption” (Shelley 1993, 43).  For children, the death of a pet is often a first experience of mortality; thus in Frankenweenie it is the dog, Sparky’s, death that allows Victor to confront the notion of perishability that so horrifies his predecessor in Shelley’s novel.  This experience of death and perishability also precipitates the story events and initiates the move into the horror genre by inspiring Victor’s act of monster-making.

The scene in which Victor reanimates Sparky provides Burton and his team with much opportunity to revel in horror movie history and to pay homage to the films of James Whale, particularly Frankenstein (1931) and its sequel, The Bride of Frankenstein (1935).  Lightning flashes and thunder crashes as Victor sews Sparky’s body back together and fixes bolts to his neck; the body is then covered by a sheet and raised through the roof to receive the life-giving electric charge.  Yet here, too, the child’s eye view is at work.  Attentive viewers will notice that Sparky’s body is laid out on an ironing board, and that toys, appliances, and other household objects form part of the elaborate life-giving apparatus.  Signifiers of “childhood” and “ordinariness” are thus interwoven with the signifiers of life, creation, and monstrosity borrowed from Whale.  Instead of fingers twitching and eyes opening, Sparky’s “alive-ness” is signified by a wagging tail; and instead of proclaiming “It’s alive!” like his predecessor in the Whale films, young Victor Frankenstein says “You’re alive”.  This shift in language reveals that the monster has been created according to a child’s desires and wishes: the moment of creation is framed by Victor’s desire not only for Sparky to still be alive but for the friendship, happiness, and unconditional love that a pet often represents.  Accordingly, the child views the monster as a friend and companion (you) rather than as the product of an experiment (it).

This transformation of Frankenstein to suit a child’s perspective certainly involves a degree of softening, a removal of some aspects of violence and conflict that define the original tale.  For instance, Shelley’s novel and most of its adaptations are constructed around the conflict between monster and maker – this conflict is not present in Frankenweenie.  As we would expect given the film’s target audience, Burton and his screenwriter John August also de-sexualise the Frankenstein tale: another notable absence is Shelley’s sub-plot involving the creation of a mate for the monster, and the resultant murder of Victor’s bride Elizabeth on their wedding night.  This does not mean, however, that Frankenweenie shies away from an exploration of monstrosity and horror.  Indeed, while Sparky himself is not depicted as a true monster, the film is replete with images of monstrosity.  These come particularly in the form of the creatures that Victor’s classmates bring to life: pets and other icons of familiarity, domesticity, and innocence (sea monkeys, a fluffy white cat, a dead hamster) who become snarling, terrifying, rampaging beasts.  The image of these monsters running amok through the fairground of New Holland encapsulates the film’s transformation of its source material.  This scene is only tenuously connected to Shelley’s plot, yet it resounds with Frankensteinian questions and dilemmas, particularly as they might be understood by children: When you have created your monster, what are you going to do with him/her/it?  And what happens if your monster (your game, your story) escapes your control?  While exploring the lighter side of monster-making, then, the film also explores the darker side of play, re-interpreting the Frankensteinian themes of creativity, perishability, and the life/death boundary so that they are seen from a child’s perspective.

Paratexts, intertexts, and the complex world of Frankenweenie

From Frankenweenie: an Electrifying Book. One of the many examples of the process of stop-motion animation and the making of Sparky.

From Frankenweenie: an Electrifying Book. One of the many examples of the process of stop-motion animation and the making of Sparky.

While not a notable box office success, Frankenweenie received a generally positive critical reception.  The film is rated highly – at 87% – on the aggregate review website Rotten Tomatoes, and is frequently described by reviewers as an enjoyable product for both children and adults (see, for instance, Paatsch 2012; Chang 2012; Mazmanian 2012).  Occasionally, charges of simplification are levelled at the film: Peter Bradshaw in The Guardian describes Frankenweenie as “a sentimental kind of retro gothic lite, appearing under the Disney banner” (2012), while A.O. Scott in the New York Times writes that “the movie, a Walt Disney release, also feels tame and compromised” (2012).  Other reviewers found the film dark enough to be entertaining, with many making positive mention of Burton’s ability to balance the sweetness of a children’s story with the darkness of a horror film.  Leigh Paatsch in the Herald Sun, for instance, commends the film for “deftly balancing blatant eeriness with a chipper cheeriness that excuses many a macabre event” (2012).  Lou Lumenick in the New York Post praises the film for its “creepy but basically sweet humor” (2012), as does Matthew Bond in the Daily Mail Australia who describes it as “strange, but also touching and lovely” (2012).  In Time magazine, Richard Corliss addresses the film’s boundary-crossing quality when he notes approvingly that “Frankenweenie’s message to the young” is that “children should play with dead things” (2012).

This positive reception sets Burton’s film apart from other recent children’s films that play with horror tropes and characters, such as the aforementioned Hotel Transylvania and Igor, both of which received lukewarm reviews.  Hotel Transylvania in particular was frequently criticised for its shallow approach to the narratives it draws upon, including Shelley’s Frankenstein (see, for instance, Reynolds 2012; Collin 2012).  L. Kent Wolgamott in the Lincoln Journal Star (2012) observes that while Frankenweenie did not perform as well at the box office as Hotel Transylvania, it is “by far, the superior film” (and he contextualises this comment by urging readers not to “consider box-office returns to be the only measure of a film’s success”, adding that with Frankenweenie Burton has created a “masterpiece”).

Some reviews of Frankenweenie mention the construction of a child’s eye view.  Adam Mazmanian in The Washington Times, for instance, identifies this as the means by which the film “draw[s] in young audiences”, adding that its “knowing winks at horror-movie history will appeal to grown-ups” (2012).  It is interesting that Mazmanian feels the need to separate the film’s audience into these two distinct categories, and that he distinguishes the “adult” and “child” sections of the audience by an ability (or lack thereof) to “get” the film’s intertextual references.  Wolgamott takes this further, praising the film for its references to classic horror movies but adding “that’s not anything the preschool through middle school animation crowd is going to get, or could possibly care about” (2012).  Both critics agree that intertextuality is a means by which Frankenweenie resists simplification and becomes something more than a light and fluffy children’s film.  At the same time, both critics produce distinct readings of the film’s child and adult audiences, and locate the qualities of media literacy and cultural awareness (which might enable the decoding of the film’s intertextuality) squarely within the adult space.

It is certainly true that Frankenweenie is littered with intertextual references: to other texts in the Frankenstein mythos (particularly the films of James Whale), to films in Burton’s oeuvre (such as Edward Scissorhands), and to texts in the horror genre more broadly (such as the Japanese monster movie Gamera).  This is coupled with a playful self-reflexivity that we often see in filmic adaptations of Frankenstein.  As Esther Schor (2003) has pointed out, adaptations of Shelley’s novel – from the early stage productions to the first known Frankenstein film in 1910, and beyond – often depict the monster’s coming-to-life in a spectacular and self-referential way; most filmic versions, in particular, play upon what William Nestrick (1979, 292) has termed the “myth of animation” – a thematic link or bridge between the Frankenstein tale and cinema’s own powers to bring a still image, body, or scene to life.  Frankenweenie, of course, is an animated film, and this brings new meaning to Nestrick’s “myth”.  The technologies of movie-making and, specifically, stop-motion animation are spectacularised in the image of Sparky’s coming-to-life, adding another layer of intertextuality to a film already rich with cultural references.

It may be tempting to assume, as do Mazmanian and Wolgamott, that children are excluded from this intertextual conversation.  Indeed, it has become increasingly common for children’s films to engage in a dual mode of address, enchanting children with stories, songs, and imagery while offering jokes, intertextual references, or clever moments of self-awareness to adults.  The implication is that long-suffering parents should be rewarded for watching films with their children or otherwise lured into the watching process by the promise of adult-centric entertainment.  Burton’s film is somewhat different because the intertextual references are closely bound to the narrative – they are less an amusing aside for adults than part of the film’s very fabric.  They are also, potentially, a means of encouraging audiences to connect with the source material.  While Frankenweenie does not openly strive to generate reverence for (or even awareness of) Mary Shelley, her novel, and the act of reading Frankenstein, it arguably promotes a more complex form of literacy that speaks directly to the process of adaptation itself.  By referencing the movies of James Whale, in particular, Burton positions his film within a web of Frankenstein texts and also destabilises the primacy of Shelley’s novel as source text: adapting Frankenstein, we are told, is a complex business that involves the engagement with already apparent intertextuality rather than the “recovery” of a single source text from out of the depths of adaptation history.

It is likely, furthermore, that many of the children who constitute Frankenweenie’s primary audience are able to decode the film’s intertextual references due to their familiarity with the horror genre and its tropes, characters, and conventions.  As noted above, children have traditionally been locked out of the horror genre; in recent years, however, encounters between young audiences and horror have been initiated through a plethora of child-friendly horror texts: as well as Hotel Transylvania and Igor, these include the films ParaNorman (Sam Fell and Chris Butler, 2012) and Monsters vs Aliens (Rob Letterman and Conrad Vernon, 2009), the video game Plants vs Zombies (PopCap Games, 2009), the books and television series Grossology (Sylvia Branzei, 1992-1997; Nelvana Limited, 2006-2009), and Chris Riddell’s Goth Girl books (2013-2015), as well as older but still relevant texts such as The Simpsons (which frequently lampoons the genre through its “Treehouse of Horror” episodes).  Meanwhile, imagery and tropes from the Frankenstein tale have been so pervasively circulated in popular culture for such a long time that children are likely to have some degree of familiarity with the tale even if they do not connect it to its original source.  Indeed, it is not uncommon for children’s texts to make passing reference to the tale and its characters (for instance, an episode of the cartoon Spongebob Squarepants is entitled “Frankendoodle”, while Dav Pilkey’s Captain Underpants books contain a character named “Frankenbooger”).

It is also likely that children today are familiar with complex levels of intertextuality and are adept at negotiating intersecting currents of media; thus Cathlena Martin writes of the “overlapping intertextual nature of children’s culture” (2009, 86).  In her analysis of the transmedia adaptation of the novel Charlotte’s Web, Martin claims that an enjoyment and understanding of intertextuality may come more naturally to today’s children, who “experience transmedia stories on a regular basis” and therefore “no longer view the printed text as the only way to experience [a literary classic such as] Charlotte’s Web”, whereas adults are more likely to “resist multi-media adaptation, relying on the supremacy of print text as ‘high art’” (2009, 88).  This returns us to the concept of “fidelity” to an original text, and suggests that in discussions of adaptation for children fidelity is likely to be a concept imposed by adult readers and critics rather than something inherently understood or valued by children.  If this raises concerns over the disappearance of “the book” as a cultural object, it also demonstrates that “simplicity” is not a concept that sits well with the highly interconnected, transmedia quality of children’s culture today.

The promotional release of the free Frankenweenie: an Electrifying Book. The e-book explores the production of Frankenweenie: readers are given access to production photographs, original artwork, and interviews. This is a promotional mock poster for a film titled "Return of the Vampire Cat".

The promotional release of the free Frankenweenie: an Electrifying Book. The e-book explores the production of Frankenweenie: readers are given access to production photographs, original artwork, and interviews. This is a promotional mock poster for a film titled “Return of the Vampire Cat”.

Interestingly, the promotional material for Frankenweenie played upon this ability in young audiences to understand and enjoy intertextuality.  Elliott reminds us that “[t]ie-in merchandise produces and distributes the culture of Disney beyond the cinema” (2014, 195); yet the marketing campaign for Frankenweenie took a very different route from the usual toys, games, and Happy Meals associated with Disney and with the process of Disneyfication.  Instead, the film was promoted through such unusual means as the release of six mock B-movie posters each featuring one of the child characters together with the monster he/she creates (including Night of the Were-Rat: a Tale of Terror featuring “Edgar E. Gore” and Return of the Vampire Cat featuring “Weird Girl”).  These promotional texts not only foreground the film’s child protagonists (as opposed to its adult characters) but serve to locate “childhood” within the parameters of the horror genre and within monster-movie history.  If entryway paratexts guide and instruct our viewing of a media text, as Gray (2010) suggests, these posters invite us to connect childhood with monstrosity in a way that “preps” us for the viewing of Frankenweenie itself (whether we are adults or children).  They also underscore the overall playfulness of the film and relatedly its resistance to the processes of Disneyfication and simplification.  Due to the foregrounding of the child characters, furthermore, the posters specifically address child audiences and clearly include them in the film’s intertextual conversation.

Another key aspect of the film’s promotion was the release of a free e-book entitled Frankenweenie: an Electrifying Book.  Designed for audiences of all ages, the e-book explores the production of Frankenweenie: readers are given access to production photographs, original artwork, and interviews, with particular emphasis on the process of stop-motion animation and the making of Sparky (who we can view as a sketch, a 3-D model, and a finished “product”).  In this way, the e-book allows children access to Nestrick’s “myth of animation” and to the idea of animation as a “bridge” between the narrative and the technology of Frankenweenie.  The e-book also makes the film’s intertextuality more evident.  It begins, for instance, with a foreward by actor Martin Landau accompanied an image of the character he voices (Mr Rzykruski); Landau discusses his previous collaboration with Tim Burton, the film Ed Wood, and his role in this film as Bela Lugosi, star of the horror classic Dracula (Tod Browning, 1931).  A pop-up button informs us that Landau’s character is also “a nod to Vincent Price, the late actor known for his iconic roles in various horror films” (Disney Book Group, 2012).  The e-book thus enables or enhances the ability of any audience member (including children) to decode the film’s intertextual references – and even, arguably, leads young audiences back to the various source texts that inspired Frankenweenie.

In this way, the film (together with its promotional material) both assumes and encourages a level of cultural literacy regarding the Frankenstein tale and, more broadly, the horror genre itself.  As this analysis has demonstrated, the film’s intertextuality works together with its paratexts to cultivate an awareness of what lies beyond its own textual boundaries.  Frankenweenie thus imagines and constructs its audience to be a media-literate and curious child.

Conclusion

Prior to the release of Tim Burton’s Frankenweenie, the thought of an animated film based on Mary Shelley’s Frankenstein and released under the Disney banner might have horrified literary purists and fans of horror cinema alike.  An animated Frankenstein, in which darkness and moral conflict are replaced by cute animal side-kicks and catchy songs, may well have been taken as a sign of Disney’s cultural domination and its ability not just to appropriate literary material but to colonise sites of literary and cultural meaning.  Burton’s film, however, demonstrates that “Disneyfication” is not the only route to adapting a literary classic for children, and that the transformation of such a tale within the space of a child’s worldview need not involve a simplification process.  As noted above, we can contextualise Frankenweenie within a recent trend in media and popular culture that has seen the horror genre re-imagined for young audiences; yet Burton’s film can be read not just as an example of “horror for kids” but as a startlingly successful transformation of a previously inaccessible tale in line with the concerns that define a child’s world.  Importantly, Frankenweenie’s most powerful images are not cartoonish renditions of monsters and mad scientists – they are the images of Victor grieving for Sparky, and of the neighbourhood kids struggling to control the monsters they have unleashed.  These themes of loss, and of losing control, are central to the film’s re-imagining of a classic horror tale according to a child’s eye view.  In this way, Frankenweenie makes Frankenstein accessible to children and also gives adult viewers a sense of what horror, otherness, and monstrosity could mean to a child.

 

Works cited

Annunziato, Sarah.  2014.  “A Child’s Eye View of Where the Wild Things Are: Lessons from Spike Jonze’s Film Adaptation of Maurice Sendak’s Picture Book.”  Journal of Children and Media 8 (3): 253-266.

Bell, Elizabeth, Lynda Haas, and Laura Sells.  1995.  “Introduction: Walt’s in the Movies”.  In From Mouse to Mermaid: The Politics of Film, Gender, and Culture, edited by Elizabeth Bell, Lynda Haas, and Laura Sells, 1-17.  Bloomington and Indianapolis: Indiana University Press.

Bond, Matthew.  2012.  “Dr Burton’s gothic horror has a heart: Frankenweenie is touching and enjoyable”.  Daily Mail Australia, October 23.  Accessed August 6, 2014.  http://www.dailymail.co.uk/tvshowbiz/article-2221392/Frankenweenie-movie-review-Tim-Burtons-gothic-horror-heart.html

Bradshaw, Peter.  2012.  “Frankenweenie: first look”.  The Guardian, October 11.  Accessed August 6, 2014.  http://www.theguardian.com/film/2012/oct/10/frankenweenie-review-london-film-festival-tim-burton

Brooker, Will.  2012.  Hunting the Dark Knight: Twenty-first Century Batman.  London and New York: I.B. Tauris.

Buckingham, David.  1996.  Moving Images: Understanding Children’s Emotional Responses to Television.  Manchester and New York: Manchester University Press.

Chang, Justin.  2012.  “Review: Frankenweenie”.  Variety, September 20.  Accessed August 6, 2014.  http://variety.com/2012/film/reviews/frankenweenie-1117948379/

Collin, Robbie.  2012.  “Hotel Transylvania review”.  The Telegraph, October 11.  Accessed August 6, 2014.  http://www.telegraph.co.uk/culture/film/filmreviews/9602345/Hotel-Transylvania-review.html

Corliss, Richard.  2012.  “Tim Burton’s Frankenweenie Movie Review: A Re-animated Delight”.  Time, October 4.  Accessed August 6, 2014.  http://entertainment.time.com/2012/10/04/tim-burtons-frankenweenie-a-re-animated-delight/

Elliott, Kamilla.  2014.  “Tie-Intertextuality, or, Intertextuality as Incorporation in the Tie-in Merchandise to Disney’s Alice in Wonderland (2010)”.  Adaptation 7 (2): 191-211.

Frankenweenie: an Electrifying Book.  2012.  Disney Book Group.  Accessed August 20, 2014.  Available at https://itunes.apple.com/au/book/frankenweenie-electrifying/id557041056?mt=11

Gaiman, Neil.  2006.  “The Bride of Frankenstein”.  Neil Gaiman (official website).  Accessed 14 June 2013. http://www.neilgaiman.com/p/Cool_Stuff/Essays/Essays_By_Neil/The_Bride_of_Frankenstein.  From Cinema Macabre, edited by Mark Morris.  Hornsea: PS.

Gray, Jonathon.  2010.  Show Sold Separately: Promos, Spoilers, and Other Media Paratexts.  New York: New York University Press.

Hastings, A. Waller.  1993.  “Moral Simplification in Disney’s The Little Mermaid”.  The Lion and the Unicorn 17 (1): 83-92.

Johnson, Barbara.  1982.  “My Monster/My Self”.  Diacritics 12: 2-10.

Lavalley, Albert J.  1979.  “The Stage and Film Children of Frankenstein: A Survey”.  In The Endurance of Frankenstein: Essays on Mary Shelley’s Novel, edited by George Levine and U.C. Knoeplfmacher, 243-289.  Berkeley: University of California Press.

Leitch, Thomas.  2003.  “Twelve Fallacies in Contemporary Adaptation Theory”.  Criticism 45 (2): 149-171.

Leitch, Thomas.  2007.  Film Adaptation and its Discontents: From Gone with the Wind to the Passion of the Christ.  Baltimore: John Hopkins University Press.

Lemire, Christy.  2012.  “Frankenweenie Review: Tim Burton Reminds Us Why We Love Him”.  The Huffington Post, October 2.  Accessed August 6, 2014.  http://www.huffingtonpost.com/2012/10/03/frankenweenie-review-tim-burton_n_1935142.html

Lumenick, Lou.  2012.  “‘Frankenweenie is a Monster Piece!”.  The New York Post, October 5.  Accessed August 6, 2014.  http://nypost.com/2012/10/05/frankenweenie-is-a-monster-piece/

Martin, Cathlena.  2009.  “Charlotte’s Website: Media Transformation and the Intertextual Web of Children’s Culture”.  In Adaptation in Contemporary Culture: Textual Infidelities, edited by Rachel Carroll, 85-95.  London: Continuum.

Mazmanian, Adam.  2012.  “Movie Review: Frankenweenie”.  The Washington Times, October 4.  Accessed August 6, 2014. http://www.washingtontimes.com/news/2012/oct/4/movie-review-frankenweenie/#ixzz37OlxdQap.

Mellor, Anne K.  2003.  “‘Making a Monster’: an introduction to Frankenstein”.  In The Cambridge Companion to Mary Shelley, edited by Esther Schor, 9-25.  Cambridge: Cambridge University Press.

Napier, Susan J.  2000.  Anime: From Akira to Princess Mononoke – Experiencing Contemporary Japanese Animation.  New York: Palgrave.

Napolitano, Marc.  2009.  “Disneyfying Dickens: Oliver & Company and The Muppet Christmas Carol as Dickensian Musicals”.  Studies in Popular Culture 32 (1): 79-102.

 Nestrick, William.  1979.  “Coming to Life: Frankenstein and the Nature of Film Narrative”.  In The Endurance of Frankenstein: Essays on Mary Shelley’s Novel, edited by George Levine and U.C. Knoeplfmacher, 290-315.  Berkeley: University of California Press.

Paatsch, Leigh.  2012.  “Film review: Frankenweenie Enchants Adults Too”.  Herald Sun, October 25.  Accessed August 6, 2014.  http://www.heraldsun.com.au/entertainment/movies/frankenweenie-enchants-adults-too/story-e6frf8r6-1226503055322

Reynolds, Simon.  2012.  “Hotel Transylvania Review”.  Digital Spy, October 9.  Accessed August 6, 2014.  http://www.digitalspy.com.au/movies/review/a422323/hotel-transylvania-review.html#~oM8bcdFfwKA0yf

Sayers, Frances Clarke.  1965.  “Walt Disney Accused”.  Horn Book 41: 602-611.

Schor, Esther.  2003.  “Frankenstein and Film”.  In The Cambridge Companion to Mary Shelley, edited by Esther Schor, 63-83.  Cambridge: Cambridge University Press.

Scott, A.O.  2012.  “It’s Aliiiive! And Wagging Its Tail – ‘Frankenweenie,’ Tim Burton’s Homage to Horror Classics”.  The New York Times, October 4.  Accessed August 6, 2014.  http://www.nytimes.com/2012/10/05/movies/frankenweenie-tim-burto…age-to-horror-classics.html?smid=tw-nytimesmovies&seid=auto&_r=0

Semenza, Gregory M. Colón.  2008.  “Teens, Shakespeare, and the Dumbing Down Cliché: The Case of The Animated Tales”.  Shakespeare Bulletin 26 (2): 37-68.

Shelley, Mary.  1993.  Frankenstein, or, The Modern Prometheus.  Hertfordshire: Wordsworth Editions.

Smith, Sarah J.  2005.  Children, Cinema and Censorship: from Dracula to the Dead End Kids.  London and New York: I.B. Tauris.

Stam, Robert.  2005.  “Introduction: the Theory and Practice of Adaptation”.  In Literature and Film: A Guide to the Theory and Practice of Film Adaptation, edited by Robert Stam and Alessandra Raengo, 1-52.  Malden, MA: Blackwell.

Townshend, Dale.  2008.  “The Haunted Nursery: 1764-1830”.  In The Gothic in Children’s Literature: Haunting the Borders, edited by Anna Jackson, Karen Coats, and Roderick McGillis, 15-38.  London and New York: Routledge.

Waldby, Catherine.  2002.  “The Instruments of Life: Frankenstein and Cyberculture”.  In Prefiguring Cyberculture: An Intellectual History, edited by Darren Tofts, Annemarie Jonson and Alessio Cavallaro, 28-37.  Cambridge and London: MIT Press.

Wells, Paul.  1998.  Understanding Animation.  New York: Routledge.

Wells, Paul.  2007.  “Classic Literature and Animation: All Adaptations are Equal, but Some are More Equal Than Others”.  In The Cambridge Companion to Literature on Screen, edited by Deborah Cartmell and Imelda Whelehan, 199-211.  Cambridge and New York: Cambridge University Press.

Whelehan, Imelda.  1999.  “Adaptations: the Contemporary Dilemmas”.  In Adaptations: From Text to Screen, Screen to Text, edited by Deborah Cartmell and Imelda Whelehan, 3-19.  London and New York: Routledge.

Wolgamott, L. Kent.  2012.  “Frankenweenie a box-office bomb, but superior film”.  Lincoln Journal Star, October 10.  Accessed August 6, 2014.  http://journalstar.com/entertainment/movies/l-kent-wolgamott-fran…b-but-superior/article_42409e82-89b9-5794-8082-7b5de3d469e2.html.=

Zipes, Jack.  1993.  “The Trials and Tribulations of Little Red Riding Hood”.  In The Trials and Tribulations of Little Red Riding Hood, edited by Jack Zipes, 17-88.  London and New York: Routledge.

Zipes, Jack.  1995.  “Breaking the Disney Spell”.  In From Mouse to Mermaid: The Politics of Film, Gender, and Culture, edited by Elizabeth Bell, Lynda Haas, and Laura Sells, 21-42.  Bloomington and Indianapolis: Indiana University Press.

 

Films cited

Burton, Tim.  2012.  Frankenweenie.  USA: Walt Disney Pictures.

Leondis, Anthony.  2008.  Igor.  USA: Roadshow Entertainment.

Tartakovsky, Genndy.  2012.  Hotel Transylvania.  USA: Columbia Pictures.

Whale, James.  1931.  Frankenstein.  USA: Universal Pictures.

Whale, James.  1935.  The Bride of Frankenstein.  USA: Universal Pictures.

 

Bio:

Erin Hawley teaches in the Journalism, Media, and Communications program at the University of Tasmania.  Her current research interests include children’s media culture, adaptation, and media education.

Read, Watch, Listen: A commentary on eye tracking and moving images – Tim J. Smith

Abstract

Eye tracking is a research tool that has great potential for advancing our understanding of how we watch movies. Questions such as how differences in the movie influences where we look and how individual differences between viewers alters what we see can be operationalised and empirically tested using a variety of eye tracking measures. This special issue collects together an inspiring interdisciplinary range of opinions on what eye tracking can (and cannot) bring to film and television studies and practice. In this article I will reflect on each of these contributions with specific focus on three aspects: how subtitling and digital effects can reinvigorate visual attention, how audio can guide and alter our visual experience of film, and how methodological, theoretical and statistical considerations are paramount when trying to derive conclusions from eye tracking data.

 

Introduction

I have been obsessed with how people watch movies since I was a child. All you have to do is turn and look at an audience member’s face at the movies or at home in front of the TV to see the power the medium holds over them. We sit enraptured, transfixed and immersed in the sensory patterns of light and sound projected back at us from the screen. As our physical activity diminishes our mental activity takes over. We piece together minimal audiovisual cues to perceive rich otherworldly spaces, believable characters and complex narratives that engage us mentally and move us emotionally. As I progressed through my education in Cognitive Science and Psychology I was struck by how little science understood about cinema and the mechanisms filmmakers used to create this powerful experience.[i] Reading the film literature, listening to filmmakers discuss their craft and excavating gems of their craft knowledge I started to realise that film was a medium ripe for psychological investigation. The empirical study of film would further our understanding of how films work and how we experience them but it would also serve as a test bed for investigating complex aspects of real-world cognition that were often considered beyond the realms of experimentation. As I (Smith, Levin & Cutting, 2010) and others (Anderson, 2006) have argued elsewhere, film evolved to “piggy back” normal cognitive development and use basic cognitive tendencies such as attentional preferences, theory of mind, empathy and narrative structuring of memory to make the perception of film as enjoyable and effortless as possible. By investigating film cognition we can, in turn advance our understanding of general cognition. But to do so we need to step outside of traditional disciplinary boundaries concerning the study of film and approach the topic from an interdisciplinary perspective. This special issue represents a highly commendable attempt to do just that.

By bringing together psychologists, film theorists, philosophers, vision scientists, neuroscientists and screenwriters this special issue (and the Melbourne research group that most contributors belong to) provides a unique perspective on film viewing. The authors included in this special issue share my passion for understanding the relationship between viewers and film but this interest manifests in very different ways depending on their perspectives (see Redmond, Sita, and Vincs, this issue; for a similar personal journey into eye tracking as that presented above). By focussing on viewer eye movements the articles in this special issue provide readers from a range of disciplines a way into the eye tracking investigation of film viewing. Eye tracking (as comprehensively introduced and discussed by Dyer and Pink, this issue) is a powerful tool for quantifying a viewer’s experience of a film, comparing viewing behaviour across different viewing conditions and groups as well as testing hypotheses about how certain cinematic techniques impact where we look. But, as is rightly highlighted by several of the authors in this special issue eye tracking is not a panacea for all questions about film spectatorship.

Like all experimental techniques it can only measure a limited range of psychological states and behaviours and the data it produces does not say anything in and of itself. Data requires interpretation. Interpretation can take many forms[ii] but if conclusions are to be drawn about how the data relates to psychological states of the viewer this interpretation must be based on theories of psychology and ideally confirmed using secondary/supporting measures. For example, the affective experience of a movie is a critical aspect which cognitive approaches to film are often wrongly accused of ignoring. Although, cognitive approaches to film often focus on how we comprehend narratives (Magliano and Zacks, 2011), attend to the image (Smith, 2013) or follow formal patterns within a film (Cutting, DeLong and Nothelfer, 2010) several cognitivists have focussed in depth on emotional aspects (see the work of Carl Plantinga, Torben Grodal or Murray Smith). Eye tracking is the perfect tool for investigating the impact of immediate audiovisual information on visual attention but it is less suitable for measuring viewer affect. Psychophysiological measures such as heart rate and skin conductance, neuroimaging methods such as fMRI or EEG, or even self-report ratings may be better for capturing a viewer’s emotional responses to a film as has been demonstrated by several research teams (Suckfull, 2000; Raz et al, 2014). Unless the emotional state of the viewer changed where they looked or how quickly they moved their eyes the eye tracker may not detect any differences between two viewers with different emotional states.[iii]

As such, a researcher interested in studying the emotional impact of a film should either choose a different measurement technique or combine eye tracking with another more suitable technique (Dyer and Pink, this issue). This does not mean that eye tracking is unsuitable for studying the cinematic experience. It simply means that you should always choose the right tool for the job and often this means combining multiple tools that are strong in different ways. As Murray Smith (the current President of the Society for the Cognitive Study of the Moving Images; SCSMI) has argued, a fully rounded investigation of the cinematic experience requires “triangulation” through the combination of multiple perspectives including psychological, neuroscientific and phenomenological/philosophical theory and methods (Smith, 2011) – an approach taken proudly across this special issue.

For the remainder of my commentary I would like to focus on certain themes that struck me as most personally relevant and interesting when reading the other articles in this special issue. This is by no means an exhaustive list of the themes raised by the other articles or even an assessment of the importance of the particular themes I chose to select. There are many other interesting observations made in the articles I do not focus on below but given my perspective as a cognitive scientist and current interests I decided to focus my commentary on these specific themes rather than make a comprehensive review of the special issues or tackle topics I am unqualified to comment on. Also, I wanted to take the opportunity to dispel some common misconceptions about eye tracking (see the section ‘Listening to the data’) and empirical methods in general.

Reading an image

One area of film cognition that has received considerable empirical investigation is subtitling. As Kruger, Szarkowska and Krejtz (this issue) so comprehensively review, they and I believe eye tracking is the perfect tool for investigating how we watch subtitled films. The presentation of subtitles divides the film viewing experience into a dual- task: reading and watching. Given that the media was originally designed to communicate critical information through two channels, the image and soundtrack introducing text as a third channel of communication places extra demands on the viewer’s visual system. However, for most competent readers serially shifting attention between these two tasks does not lead to difficulties in comprehension (Kruger, Szarkowska and Krejtz, this issue). Immediately following the presentation of the subtitles gaze will shift to the beginning of the text, saccade across the text and return to the centre of interest within a couple of seconds. Gaze heatmaps comparing the same scenes with and without subtitles (Kruger, Szarkowska and Krejtz, this issue; Fig. 3) show that the areas of the image fixated are very similar (ignoring the area of the screen occupied by the subtitles themselves) and rather than distracting from the visual content the presence of subtitles seems to actually condense the gaze behaviour on the areas of central interest in an image, e.g. faces and the centre of the image. This illustrates the redundancy of a lot of the visual information presented in films and the fact that under non-subtitle conditions viewers rarely explore the periphery of the image (Smith, 2013).

My colleague Anna Vilaró and I recently demonstrated this similarity in an eye tracking study in which the gaze behaviour of viewers was compared across versions of an animated film, Disney’s Bolt (Howard & Williams, 2008) either in the original English audio condition, a Spanish language version with English subtitles, an English language version with Spanish subtitles and a Spanish language version without subtitles (Vilaró, & Smith, 2011). Given that our participants were English speakers who did not know Spanish these conditions allowed us to investigate both where they looked under the different audio and subtitle conditions but also what they comprehended. Using cued recall tests of memory for verbal and visual content we found no significant differences in recall for either types of content across the viewing conditions except for verbal recall in the Spanish-only condition (not surprisingly given that our English participants couldn’t understand the Spanish dialogue). Analysis of the gaze behaviour showed clear evidence of subtitle reading, even in the Spanish subtitle condition (see Figure 1) but no differences in the degree to which peripheral objects were explored. This indicates that even when participants are watching film sequences without subtitles and know that their memory will be tested for the visual content their gaze still remains focussed on central features of a traditionally composed film. This supports arguments for subtitling movies over dubbing as, whilst placing greater demands on viewer gaze and a heightened cognitive load there is no evidence that subtitling leads to poorer comprehension.

Figure 1: Figure from Vilaró & Smith (2011) showing the gaze behaviour of multiple viewers directed to own language subtitles (A) and foreign language/uninterpretable subtitles (B).

Figure 1: Figure from Vilaró & Smith (2011) showing the gaze behaviour of multiple viewers directed to own language subtitles (A) and foreign language/uninterpretable subtitles (B).

The high degree of attentional synchrony (Smith and Mital, 2013) observed in the above experiment and during most film sequences indicates that all visual features in the image and areas of semantic significance (e.g. social information and objects relevant to the narrative) tend to point to the same part of the image (Mital, Smith, Hill and Henderson, 2011). Only when areas of the image are placed in conflict through image composition (e.g. depth of field, lighting, colour or motion contrast) or staging (e.g. multiple actors) does attentional synchrony break down and viewer gaze divide between multiple locations. Such shots are relatively rare in mainstream Hollywood cinema or TV (Salt, 2009; Smith, 2013) and when used the depicted action tends to be highly choreographed so attention shifts between the multiple centres of image in a predictable fashion (Smith, 2012). If such choreographing of action is not used the viewer can quickly exhaust the information in the image and start craving either new action or a cut to a new shot.

Hochberg and Brooks (1978) referred to this as the visual momentum of the image: the pace at which visual information is acquired. This momentum is directly observable in the saccadic behaviour during an images presentation with frequent short duration fixations at the beginning of a scene’s presentation interspersed by large amplitude saccades (known as the ambient phase of viewing; Velichovsky, Dornhoefer, Pannasch and Unema, 2000) and less frequent, longer duration fixations separated by smaller amplitude saccades as the presentation duration increases (known as the focal phase of viewing; Velichovsky et al., 2000). I have recently demonstrated the same pattern of fixations during viewing of dynamic scenes (Smith and Mital, 2013) and shown how this pattern gives rise to more central fixations at shot onset and greater exploration of the image and decreased attentional synchrony as the shot duration increases (Mital, Smith, Hill and Henderson, 2011). Interestingly, the introduction of subtitles to a movie may have the unintended consequence of sustaining visual momentum throughout a shot. The viewer is less likely to exhaust the information in the image because their eyes are busy saccading across the text to acquire the information that would otherwise be presented in parallel to the image via the soundtrack. This increased saccadic activity may increase the cognitive load experienced by viewers of subtitled films and change their affective experience, producing greater arousal and an increased sense of pace.

For some filmmakers and producers of dynamic visual media, increasing the visual momentum of an image sequence may be desirable as it maintains interest and attention on the screen (e.g. Michael Bay’s use of rapidly edited extreme Close-Ups and intense camera movements in the Transformer movies). In this modern age of multiple screens fighting for our attention when we are consuming moving images (e.g. mobile phones and computer screens in our living rooms and even, sadly increasingly at the cinema) if the designers of this media are to ensure that our visual attention is focussed on their screen over the other competing screens they need to design the visual display in a way that makes comprehension impossible without visual attention. Feature Films and Television dramas often rely heavily on dialogue for narrative communication and the information communicated through the image may be of secondary narrative importance to the dialogue so viewers can generally follow the story just by listening to the film rather than watching it. If producers of dynamic visual media are to draw visual attention back to the screen and away from secondary devices they need to increase the ratio of visual to verbal information. A simple way of accomplishing this is to present the critical audio information through subtitling. The more visually attentive mode of viewing afforded by watching subtitled film and TV may partly explain the growing interest in foreign TV series (at least in the UK) such as the popularity of Nordic Noir series such as The Bridge (2011) and The Killing (2007).

Another way of drawing attention back to the screen is to constantly “refresh” the visual content of the image by either increasing the editing rate or creatively using digital composition.[iv] The latter technique is wonderfully exploited by Sherlock (2010) as discussed brilliantly by Dwyer (this issue). Sherlock contemporised the detective techniques of Sherlock Holmes and John Watson by incorporating modern technologies such as the Internet and mobile phones and simultaneously updated the visual narrative techniques used to portray this information by using digital composition to playfully superimpose this information onto the photographic image. In a similar way to how the sudden appearance of traditional subtitles involuntarily captures visual attention and draws our eyes down to the start of the text, the digital inserts used in Sherlock overtly capture our eyes and encourage reading within the viewing of the image.

If Dwyer (this issue) had eyetracked viewers watching these excerpts she would have likely observed this interesting shifting between phases of reading and dynamic scene perception. Given that the appearance of the digital inserts produce sudden visual transients and are highly incongruous with the visual features of the background scene they are likely to involuntarily attract attention (Mital, Smith, Hill & Henderson, 2012). As such, they can be creatively used to reinvigorate the pace of viewing and strategically direct visual attention to parts of the image away from the screen centre. Traditionally, the same content may have been presented either verbally as narration, heavy handed dialogue exposition (e.g. “Oh my! I have just received a text message stating….”) or as a slow and laboured cut to close-up of the actual mobile phone so we can read it from the perspective of the character. Neither takes full advantage of the communicative potential of the whole screen space or our ability to rapidly attend to and comprehend visual information and audio information in parallel.

Such intermixing of text, digital inserts and filmed footage is common in advertisements, music videos, and documentaries (see Figure 2) but is still surprisingly rare in mainstream Western film and TV. Short-form audiovisual messages have recently experienced a massive increase in popularity due to the internet and direct streaming to smartphones and mobile devices. To maximise their communicative potential and increase their likelihood of being “shared” these videos use all audiovisual tricks available to them. Text, animations, digital effects, audio and classic filmed footage all mix together on the screen, packing every frame with as much info as possible (Figure 2), essentially maximising the visual momentum of each video and maintaining interest for as long as possible.[v] Such videos are so effective at grabbing attention and delivering satisfying/entertaining/informative experiences in a short period of time that they often compete directly with TV and film for our attention. Once we click play, the audiovisual bombardment ensures that our attention remains latched on to the second screen (i.e., the tablet or smartphone) for its duration and away from the primary screen, i.e., the TV set. Whilst distressing for producers of TV and Film who wish our experience of their material to be undistracted, the ease with which we pick up a handheld device and seek other stimulation in parallel to the primary experience may indicate that the primary material does not require our full attention for us to follow what is going on. As attention has a natural ebb-and-flow (Cutting, DeLong and Nothelfer, 2010) and “There is no such thing as voluntary attention sustained for more than a few seconds at a time” (p. 421; James, 1890) if modern producers of Film and TV want to maintain a high level of audience attention and ensure it is directed to the screen they must either rely on viewer self-discipline to inhibit distraction, reward attention to the screen with rich and nuanced visual information (as fans of “slow cinema” would argue of films like those of Bela Tarr) or utilise the full range of postproduction effects to keep visual interest high and maintained on the image, as Sherlock so masterfully demonstrates.

Figure 2: Gaze Heatmaps of participants’ free-viewing a trailer for Lego Indiana Jones computer game (left column) and the Video Republic documentary (right column). Notice how both make copious use of text within the image, as intertitles and as extra sources of information in the image (such as the head-up display in A3). Data and images were taken from the Dynamic Images and Eye Movement project (DIEM; Mital, Smith, Hill & Henderson, 2010). Videos can be found here (http://vimeo.com/6628451) and here (http://vimeo.com/2883321).

Figure 2: Gaze Heatmaps of participants’ free-viewing a trailer for Lego Indiana Jones computer game (left column) and the Video Republic documentary (right column). Notice how both make copious use of text within the image, as intertitles and as extra sources of information in the image (such as the head-up display in A3). Data and images were taken from the Dynamic Images and Eye Movement project (DIEM; Mital, Smith, Hill & Henderson, 2010). Videos can be found here (http://vimeo.com/6628451) and here (http://vimeo.com/2883321).

A number of modern filmmakers are beginning to experiment with the language of visual storytelling by questioning our assumptions of how we perceive moving images. Forefront in this movement are Ang Lee and Andy and Lana Wachowski. In Ang Lee’s Hulk (2003), Lee worked very closely with editor Tim Squyers to use non-linear digital editing and after effects to break apart the traditional frame and shot boundaries and create an approximation of a comic book style within film. This chaotic unpredictable style polarised viewers and was partly blamed for the film’s poor reception. However, it cannot be argued that this experiment was wholly unsuccessful. Several sequences within the film used multiple frames, split screens, and digital transformation of images to increase the amount of centres of interest on the screen and, as a consequence increase pace of viewing and the arousal experienced by viewers. In the sequence depicted below (Figure 3) two parallel scenes depicting Hulk’s escape from a containment chamber (A1) and this action being watched from a control room by General Ross (B1) were presented simultaneously by presenting elements of both scenes on the screen at the same time. Instead of using a point of view (POV) shot to show Ross looking off screen (known as the glance shot; Branigan, 1984) followed by a cut to what he was looking at (the object shot) both shots were combined into one image (F1 and F2) with the latter shot sliding into from behind Ross’ head (E2). These digital inserts float within the frame, often gliding behind objects or suddenly enlarging to fill the screen (A2-B2). Such visual activity and use of shots-within-shots makes viewer gaze highly active (notice how the gaze heatmap is rarely clustered in one place; Figure 3). Note that this method of embedding a POV object shot within a glance shot is similar to Sherlock’s method of displaying text messages as both the glance, i.e., Watson looking at his phone, and the object, i.e., the message, are shown in one image. Both uses take full advantage of our ability to rapidly switch from watching action to reading text without having to wait for a cut to give us the information.

Figure 3: Gaze heatmap of eight participants watching a series of shots and digital inserts from Hulk (Ang Lee, 2003). Full heatmap video is available at http://youtu.be/tErdurgN8Yg.

Figure 3: Gaze heatmap of eight participants watching a series of shots and digital inserts from Hulk (Ang Lee, 2003). Full heatmap video is available at http://youtu.be/tErdurgN8Yg.

Similar techniques have been used Andy and Lana Wachowski’s films including most audaciously in Speed Racer (2008). Interestingly, both sets of filmmakers seem to intuitively understand that packing an image with as much visual and textual information as possible can lead to viewer fatigue and so they limit such intense periods to only a few minutes and separate them with more traditionally composed sequences (typically shot/reverse-shot dialogue sequences). These filmmakers have also demonstrated similar respect for viewer attention and the difficulty in actively locating and encoding visual information in a complex visual composition in their more recent 3D movies. Ang Lee’s Life of Pi (2012) uses the visual volume created by stereoscopic presentation to its full potential. Characters inhabit layers within the volume as foreground and background objects fluidly slide around each other within this space. The lessons Lee and his editor Tim Squyers learned on Hulk (2003) clearly informed the decisions they made when tackling their first 3D film and allowed them to avoid some of the issues most 3D films experience such as eye strain, sudden unexpected shifts in depth and an inability to ensure viewers are attending to the part of the image easiest to fuse across the two eye images (Banks, Read, Allison & Watt, 2012).

Watching Audio

I now turn to another topic featured in this special issue, the influence of audio on gaze (Robinson, Stadler and Rassell, this issue). Film and TV are inherently multimodal. Both media have always existed as a combination of visual and audio information. Even early silent film was almost always presented with either live musical accompaniment or a narrator. As such, the relative lack of empirical investigation into how the combination of audio and visual input influences how we perceive movies and, specifically how we attend to them is surprising. Robinson, Stadler and Rassell (this issue) have attempted to address this omission by comparing eye movements for participants either watching the original version of the Omaha beach sequence from Steven Spielberg’s Saving Private Ryan (1998) or the same sequence with the sound removed. This film sequence is a great choice for investigating AV influences on viewer experience as the intensity of the action, the hand-held cinematography and the immersive soundscape all work together to create a disorientating embodied experience for the viewer. The authors could have approached this question by simply showing a set of participants the sequence with audio and qualitatively describing the gaze behaviour at interesting AV moments during the sequence. Such description of the data would have served as inspiration for further investigation but in itself can’t say anything about the causal contribution of audio to this behaviour as there would be nothing to compare the behaviour to. Thankfully, the authors avoided this problem by choosing to manipulate the audio.

In order to identify the causal contribution of any factor you need to design an experiment in which that factor (known as the Independent Variable) is either removed or manipulated and the significant impact of this manipulation on the behaviour of interest (known as the Dependent Variable) is tested using appropriate inferential statistics. I commend Robinson, Stadler and Rassell’s experimental design as they present such an manipulation and are therefore able to produce data that will allow them to test their hypotheses about the causal impact of audio on viewer gaze behaviour. Several other papers in this special issue (Redmond, Sita and Vincs; Batty, Perkins and Sita) discuss gaze data (typically in the form of scanpaths or heatmaps) from one viewing condition without quantifying its difference to another viewing condition. As such, they are only able to describe the gaze data, not use it to test hypotheses. There is always a temptation to attribute too much meaning to a gaze heatmap (I too am guilty of this; Smith, 2013) due to their seeming intuitive nature (i.e., they looked here and not there) but, as in all psychological measures they are only as good as the experimental design within which there are employed.[vi]

Qualitative interpretation of individual fixation locations, scanpaths or group heatmaps are useful for informing initial interpretation of which visual details are most likely to make it into later visual processing (e.g. perception, encoding and long term memory representations) but care has to be taken in falsely assuming that fixation equals awareness (Smith, Lamont and Henderson, 2012). Also, the visual form of gaze heatmaps vary widely depending on how many participants contribute to the heatmap, which parameters you choose to generate the heatmaps and which oculomotor measures the heatmap represent (Holmqvist, et al., 2011). For example, I have demonstrated that unlike during reading visual encoding during scene perception requires over 150ms during each fixation (Rayner, Smith, Malcolm and Henderson, 2009). This means that if fixations with durations less than 150ms are included in a heatmap it may suggest parts of the image have been processed which in actual fact were fixated too briefly to be processed adequately. Similarly, heatmaps representing fixation duration instead of just fixation location have been shown to be a better representation of visual processing (Henderson, 2003). Heatmaps have an immediate allure but care has to be taken about imposing too much meaning on them especially when the gaze and the image are changing over time (see Smith and Mital, 2013; and Sawahata et al, 2008 for further discussion). As eye tracking hardware becomes more available to researchers from across a range of disciplines we need to work harder to ensure that it is not used inappropriately and that the conclusions that are drawn from eye tracking data are theoretically and statistically motivated (see Rayner, 1998; and Holmqvist et al, 2013 for clear guidance on how to conduct sound eye tracking studies).

Given that Robinson, Stadler and Rassell (this issue) manipulated the critical factor, i.e., the presence of audio the question now is whether their study tells us anything new about the AV influences on gaze during film viewing. To examine the influence of audio they chose two traditional methods for expressing the gaze data: area of interest (AOI) analysis and dispersal. By using nine static (relative to the screen) AOIs they were able to quantify how much time the gaze spent in each AOI and utilise this measure to work out how distributed gaze was across all AOIs. Using these measures they reported a trend towards greater dispersal in the mute condition compared to the audio condition and a small number of significant differences in the amount of time spent in some regions across the audio conditions.

However, the conclusions we can draw from these findings are seriously hindered by the low sample size (only four participants were tested, meaning that any statistical test is unlikely to reveal significant differences) and the static AOIs that did not move with the image content. By locking the AOIs to static screen coordinates their AOI measures express the deviation of gaze relative to these coordinates, not to the image content. This approach can be informative for quantifying gaze exploration away from the screen centre (Mital, Smith, Hill and Henderson, 2011) but in order to draw conclusions about what was being fixated the gaze needs to be quantified relative to dynamic AOIs that track objects of interest on the screen (see Smith an Mital, 2013). For example, their question about whether we fixate a speaker’s mouth more in scenes where the clarity of the speech is difficult due to background noise (i.e., their “Indistinct Dialogue” scene) has previously been investigated in studies that have manipulated the presence of audio (Võ, Smith, Mital and Henderson, 2012) or the level of background noise (Buchan, Paré and Munhall, 2007) and measured gaze to dynamic mouth regions. As Robinson, Stadler and Rassell correctly predicted, lip reading increases as speech becomes less distinct or the listener’s linguistic competence in the spoken language decreases (see Võ et al, 2012 for review).

Similarly, by measuring gaze dispersal using a limited number of static AOIs they are losing considerable nuance in the gaze data and have to resort to qualitative description of unintuitive bar charts (figure 4). There exist several methods for quantifying gaze dispersal (see Smith and Mital, 2013, for review) and even open-source tools for calculating this measure and comparing dispersal across groups (Le Meur and Baccino, 2013). Some methods are as easy, if not easier to calculate than the static AOIs used in the present study. For example, the Euclidean distance between the screen centre and the x/y gaze coordinates at each frame of the movie provides a rough measure of how spread out the gaze is from the screen centre (typically the default viewing location; Mital et al, 2011) and a similar calculation can be performed between the gaze position of all participants within a viewing condition to get a measure of group dispersal.

Using such measures, Coutrot and colleagues (2012) showed that gaze dispersal is greater when you remove audio from dialogue film sequences and they have also observed shorter amplitude saccades and marginally shorter fixation durations. Although, I have recently shown that a non-dialogue sequence from Sergei Eisenstein’s Alexander Nevsky (1938) does not show significant differences in eye movement metrics when the accompanying music is removed (Smith, 2014). This difference in findings points towards interesting differences in the impact diegetic (within the depicted scene, e.g. dialogue) and non-diegetic (outside of the depicted scene, e.g. the musical score) may have on gaze guidance. It also highlights how some cinematic features may have a greater impact on other aspects of a viewer’s experience than those measureable by eye tracking such as physiological markers of arousal and emotional states. This is also the conclusion that Robinson, Stadler and Rassell come to.    

Listening to the Data (aka, What is Eye Tracking Good For?)

The methodological concerns I have raised in the previous section lead nicely to the article by William Brown, entitled There’s no I in Eye Tracking: How useful is Eye Tracking to Film Studies (this issue). I have known William Brown for several years through our attendance of the Society for Cognitive Studies of the Moving Image (SCSMI) annual conference and I have a deep respect for his philosophical approach to film and his ability to incorporate empirical findings from the cognitive neurosciences, including some references to my own work into his theories. Therefore, it comes somewhat as a surprise that his article openly attacks the application of eye tracking to film studies. However, I welcome Brown’s criticisms as it provides me with an opportunity to address some general assumptions about the scientific investigation of film and hopefully suggest future directions in which eye tracking research can avoid falling into some of the pitfalls Brown identifies.

Brown’s main criticisms of current eye tracking research are: 1) eye tracking studies neglect “marginal” viewers or marginal ways of watching movies; 2) studies so far have neglected “marginal” films; 3) they only provide “truisms”, i.e., already known facts; and 4) they have an implicit political agenda to argue that the only “true” way to study film is a scientific approach and the “best” way to make a film is to ensure homogeneity of viewer experience. I will address these criticisms in turn but before I do so I would like to state that a lot of Brown’s arguments could generally be recast as an argument against science in general and are built upon a misunderstanding of how scientific studies should be conducted and what they mean.

To respond to Brown’s first criticism that eye tracking “has up until now been limited somewhat by its emphasis on statistical significance – or, put simply, by its emphasis on telling us what most viewers look at when they watch films” (Brown, this issue; 1), I first have to subdivide the criticism into ‘the search for significance’ and ‘attentional synchrony’, i.e., how similar gaze is across viewers (Smith and Mital, 2013). Brown tells an anecdote about a Dutch film scholar who’s data had to be excluded from an eye tracking study because they did not look where the experimenter wanted them to look. I wholeheartedly agree with Brown that this sounds like a bad study as data should never be excluded for subjective reasons such as not supporting the hypothesis, i.e., looking as predicted. However, exclusion due to statistical reasons is valid if the research question being tested relates to how representative the behaviour of a small set of participants (known as the sample) are to the overall population. To explain when such a decision is valid and to respond to Brown’s criticism about only ‘searching for significance’ I will first need to provide a brief overview of how empirical eye tracking studies are designed and why significance testing is important.

For example, if we were interested in the impact sound had on the probability of fixating an actor’s mouth (e.g., Robinson, Stadler and Rassell, this issue) we would need to compare the gaze behaviour of a sample of participants who watch a sequence with the sound turned on to a sample who watched it with the sound turned off. By comparing the behaviour between these two groups using inferential statistics we are testing the likelihood that these two viewing conditions would differ in a population of all viewers given the variation within and between these two groups. In actual fact we do this by performing the opposite test: testing the probability that that the two groups belong to a single statistically indistinguishable group. This is known as the null hypothesis. By showing that there is less than a 5% chance that the null hypothesis is true we can conclude that there is a statistically significant chance that another sample of participants presented with the same two viewing conditions would show similar differences in viewing behaviour.

In order to test whether our two viewing conditions belong to one or two distributions we need to be able to express this distribution. This is typically done by identifying the mean score for each participant on the dependent variable of interest, in this case the probability of fixating a dynamic mouth AOI then calculating the mean for this measure across all participants within a group and their variation in scores (known as the standard deviation). Most natural measures produce a distribution of scores looking somewhat like a bell curve (known as the normal distribution) with most observations near the centre of the distribution and an ever decreasing number of observations as you move away from this central score. Each observation (in our case, participants) can be expressed relative to this distribution by subtracting the mean of the distribution from its score and dividing by the standard deviation. This converts a raw score into a normalized or z-score. Roughly ninety-five percent of all observations will fall within two standard deviations of the mean for normally distributed data. This means that observations with a z-score greater than two are highly unrepresentative of that distribution and may be considered outliers.

However, being unrepresentative of the group mean is insufficient motivation to exclude a participant. The outlier still belongs to the group distribution and should be included unless there is a supporting reason for exclusion such as measurement error, e.g. poor calibration of the eye tracker. If an extreme outlier is not excluded it can often have a disproportionate impact on the group mean and make statistical comparison of groups difficult. However, if this is the case it suggests that the sample size is too small and not representative of the overall population. Correct choice of sample size given an estimate of the predicted effect size combined with minimising measurement error should mean that subjective decisions do not have to be made about who’s data is “right” and who should be included or excluded.

Brown also believes that eye tracking research has so far marginalised viewers who have atypical ways of watching film, such as film scholars either by not studying them or treating them as statistical outliers and excluding them from analyses. However, I would argue that the only way to know if their way of watching a film is atypical is to first map out the distribution of how viewers typically watch films. If a viewer attended more to the screen edge than the majority of other viewers in a random sample of the population (as was the case with Brown’s film scholar colleague) this should show up as a large z-score when their gaze data is expressed relative to the group on a suitable measure such as Euclidean distance from the screen centre. Similarly, a non-native speaker of English may have appeared as an outlier in terms of how much time they spent looking at the speaker’s mouth in Robinson, Stadler and Rassell’s (this issue) study. Such idiosyncrasies may be of interest to researchers and there are statistical methods for expressing emergent groupings within the data (e.g. cluster analysis) or seeing whether group membership predicts behaviour (e.g. regression). These approaches may have not previously been applied to questions of film viewing but this is simply due to the immaturity of the field and the limited availability of the equipment or expertise to conduct such studies.

In my own recent work I have shown how viewing task influences how we watch unedited video clips (Smith and Mital, 2013), how infants watch TV (Wass and Smith, in press), how infant gaze differs to adult gaze (Smith, Dekker, Mital, Saez De Urabain and Karmiloff-Smith, in prep) and even how film scholars attend to and remember a short film compared to non-expert film viewers (Smith and Smith, in prep). Such group viewing differences are of great interest to me and I hope these studies illustrate how eye tracking has a lot to offer to such research questions if the right statistics and experimental designs are employed.

Brown’s second main criticism is that the field of eye tracking neglects “marginal” films. I agree that the majority of films that have so far been used in eye tracking studies could be considered mainstream. For example, the film/TV clips used in this special issue include Sherlock (2010), Up (2009) and Saving Private Ryan (1998). However, this limit is simply a sign of how few eye tracking studies of moving images there have been. All research areas take time to fully explore the range of possible research questions within that area.

I have always employed a range of films from diverse film traditions, cultures, and languages. My first published eye tracking study (Smith and Henderson, 2008) used film clips from Citizen Kane (1941), Dogville (2003), October (1928), Requiem for a Dream (2000), Dancer in the Dark (2000), Koyaanisqatsi (1982) and Blade Runner (1982). Several of these films may be considered “marginal” relative to the mainstream. If I have chosen to focus most of my analyses on mainstream Hollywood cinema this is only because they were the most suitable exemplars of the phenomena I was investigating such as continuity editing and its creation of a universal pattern of viewing (Smith, 2006; 2012). This interest is not because, as Brown argues, I have a hidden political agenda or an implicit belief that this style of filmmaking is the “right” way to make films. I am interested in this style because it is the dominant style and, as a cognitive scientist I wish to use film as a way of understanding how most people process audiovisual dynamic scenes.

Hollywood film stands as a wonderfully rich example of what filmmakers think “fits” human cognition. By testing filmmaker intuitions and seeing what impact particular compositional decisions have on viewer eye movements and behavioural responses I hope to gain greater insight into how audiovisual perception operates in non-mediated situations (Smith, Levin and Cutting, 2012). But, just as a neuropsychologist can learn about typical brain function by studying patients with pathologies such as lesions and strokes, I can also learn about how we perceive a “typical” film by studying how we watch experimental or innovative films. My previous work is testament to this interest (Smith, 2006; 2012a; 2012b; 2014; Smith & Henderson, 2008) and I hope to continue finding intriguing films to study and further my understanding of film cognition.

One practical reason why eye tracking studies rarely use foreign language films is the presence of subtitles. As has been comprehensively demonstrated by other authors in this special issue (Kruger, Szarkowska and Krejtz, this issue) and earlier in this article, the sudden appearance of text on the screen, even if it is incomprehensible leads to differences in eye movement behaviour. This invalidates the use of eye tracking as a way to measure how the filmmaker intended to shape viewer attention and perception. The alternatives would be to either use silent film (an approach I employed with October; Smith and Henderson, 2008), remove the audio (which changes gaze behaviour and awareness of editing; Smith & Martin-Portugues Santacreau, under review) or use dubbing (which can bias the gaze down to the poorly synched lips; Smith, Batten, and Bedford, 2014). None of these options are ideal for investigating foreign language sound film and until there is a suitable methodological solution this will restrict eye tracking studies to experimental films in a participant’s native language.

Finally, I would like to counter Brown’s assertion that eye tracking investigations of film have so far only generated “truisms”. I admit that there is often a temptation to reduce empirical findings to simplified take-home messages that only seem to confirm previous intuitions such as a bias of gaze towards the screen centre, towards speaking faces, moving objects or subtitles. However, I would argue that such messages fail to appreciate the nuance in the data. Empirical data correctly measured and analysed can provide subtle insights into a phenomenon that subjective introspection could never supply.

For example, film editors believe that an impression of continuous action can be created across a cut by overlapping somewhere between two (Anderson, 1996) and four frames (Dmytryk, 1986) of the action. However, psychological investigations of time perception revealed that our judgements of duration depend on how attention is allocated during the estimated period (Zakay and Block, 1996) and will vary depending on whether our eyes remain still or saccade during the period (Yarrow et al, 2001). In my thesis (Smith, 2006) I used simplified film stimuli to investigate the role that visual attention played in estimation of temporal continuity across a cut and found that participants experienced an overlap of 58.44ms as continuous when an unexpected cut occurred during fixation and an omission of 43.63ms as continuous when they performed a saccade in response to the cut. As different cuts may result in different degrees of overt (i.e., eye movements) and covert attentional shifts these empirical findings both support editor intuitions that temporal continuity varies between cuts (Dmytryk, 1986) whilst also explaining the factors that are important in influencing time perception at a level of precision not possible through introspection.

Reflecting on our own experience of a film suffers from the fact that it relies on our own senses and cognitive abilities to identify, interpret and express what we experience. I may feel that my experience of a dialogue sequence from Antichrist (2010) differs radically from a similar sequence from Secrets & Lies (1996) but I would be unable to attribute these differences to different aspects of the two scenes without quantifying both the cinematic features and my responses to them. Without isolating individual features I cannot know their causal contribution to my experience. Was it the rapid camera movements in Antichrist, the temporally incongruous editing, the emotionally extreme dialogue or the combination of these features that made me feel so unsettled whilst watching the scene? If one is not interested in understanding the causal contributions of each cinematic decision to an audience member’s response then one may be content with informed introspection and not find empirical hypothesis testing the right method. I make no judgement about the validity of either approach as long as each researcher understands the limits of their approach.

Introspection utilises the imprecise measurement tool that is the human brain and is therefore subject to distortion, human bias and an inability to extrapolate the subjective experience of one person to another. Empirical hypothesis testing also has its limitations: research questions have to be clearly formulated so that hypotheses can be stated in a way that allows them to be statistically tested using appropriate observable and reliable measurements. A failure at any of these stages can invalidate the conclusions that can be drawn from the data. For example, an eye tracker may be poorly calibrated resulting in an inaccurate record of where somebody was looking or it could be used to test an ill formed hypothesis such as how a particular film sequence caused attentional synchrony without having another film sequence to compare the gaze data to. Each approach has its strength and weaknesses and no single approach should be considered “better” than any other, just as no film should be considered “better” than any other film.

Conclusion

The articles collected here constitute the first attempt to bring together interdisciplinary perspectives on the application of eye tracking to film studies. I fully commend the intention of this special issue and hope that it encourages future researchers to conduct further studies using these methods to investigate research questions and film experiences we have not even conceived of. However, given that the recent release of low-cost eye tracking peripherals such as the EyeTribe[vii] tracker and the Tobii EyeX[viii] has moved eye tracking from a niche and highly expensive research tool to an accessible option for researchers in a range of disciplines, I need to take this opportunity to issue a word of warning. As I have outlined in this article, eye tracking is like any other research tool in that it is only useful if used correctly, its limitations are respected, its data is interpreted through the appropriate application of statistics and conclusions are only drawn that are based on the data in combination with a sound theoretical base. Eye tracking is not the “saviour” of film studies , nor is science the only “valid” way to investigate somebody’s experience of a film. Hopefully, the articles in this special issue and the ideas I have put forward here suggest how eye tracking can function within an interdisciplinary approach to film analysis that furthers our appreciation of film in previously unfathomed ways.

 

Acknowledgements

Thanks to Rachael Bedford, Sean Redmond and Craig Batty for comments on earlier drafts of this article. Thank you to John Henderson, Parag Mital and Robin Hill for help in gathering and visualising the eye movement data used in the Figures presented here. Their work was part of the DIEM Leverhulme Trust funded project (https://thediemproject.wordpress.com/). The author, Tim Smith is funded by EPSRC (EP/K012428/1), Leverhulme Trust (PLP-2013-028) and BIAL Foundation grant (224/12).

 

References

Anderson, Joseph. 1996. The Reality of Illusion: An Ecological Approach to Cognitive Film Theory. Southern. Illinois University Press.

Batty, Craig, Claire Perkins and Jodi Sita. 2015. “How We Came To Eye Tracking Animation: A Cross-Disciplinary Approach to Researching the Moving Image”, Refractory: a Journal of Entertainment Media, 25.

Banks, Martin S., Jenny R. Read, Robert S. Allison and Simon J. Watt. 2012. “Stereoscopy and the human visual system.” SMPTE Mot. Imag. J., 121 (4), 24-43

Bradley, Margaret M., Laura Miccoli, Miguel A. Escrig and Peter J. Lang. 2008. “The pupil as a measure of emotional arousal and autonomic activation.” Psychophysiology, 45(4), 602-607.

Branigan, Edward R. 1984. Point of View in the Cinema: A Theory of Narration and Subjectivity in Classical Film. Berlin: Mouton.

Brown, William. 2015. “There’s no I in Eye Tacking: How Useful is Eye Tracking to Film Studies?”, Refractory: a Journal of Entertainment Media, 25.

Buchan, Julie N., Martin Paré and Kevin G. Munhall. 2007. “Spatial statistics of gaze fixations during dynamic face processing.” Social Neuroscience, 2, 1–13.

Coutrot, Antoine, Nathalie Guyader, Gelu Ionesc and Alice Caplier. 2012. “Influence of Soundtrack on Eye Movements During Video Exploration”, Journal of Eye Movement Research 5, no. 4.2: 1-10.

Cutting, James. E., Jordan E. DeLong and Christine E. Nothelfer. 2010. “Attention and the evolution of Hollywood film.” Psychological Science, 21, 440-447.

Dwyer, Tessa. 2015. “From Subtitles to SMS: Eye Tracking, Texting and Sherlock”, Refractory: a Journal of Entertainment Media, 25.

Dyer, Adrian. G and Sarah Pink. 2015. “Movement, attention and movies: the possibilities and limitations of eye tracking?”, Refractory: a Journal of Entertainment Media, 25.

Dmytryk, Edward. 1986. On Filmmaking. London, UK: Focal Press.

Henderson, John. M., 2003. “Human gaze control during real-world scene perception.” Trends in Cognitive Sciences, 7, 498-504.

Hochberg, Julian and Virginia Brooks. 1978). “Film Cutting and Visual Momentum”. In John W. Senders, Dennis F. Fisher and Richard A. Monty (Eds.), Eye Movements and the Higher Psychological Functions (pp. 293-317). Hillsdale, NJ: Lawrence Erlbaum.

Holmqvist, Kenneth, Marcus Nyström, Richard Andersson, Richard Dewhurst, Halszka Jarodzka and Joost van de Weijer. 2011. Eye Tracking: A comprehensive guide to methods and measures. Oxford, UK: OUP Press.

James, William. 1890. The principles of psychology (Vol.1). New York: Holt

Kruger, Jan Louis, Agnieszka Szarkowska and Izabela Krejtz. 2015. “Subtitles on the Moving Image: An Overview of Eye Tracking Studies”, Refractory: a Journal of Entertainment Media, 25.

Le Meur, Olivier and Baccino, Thierry. 2013. “Methods for comparing scanpaths and saliency maps: strengths and weaknesses.” Behavior research methods, 45(1), 251-266.

Magliano, Joseph P. and Jeffrey M. Zacks. 2011. “The Impact of Continuity Editing in Narrative Film on Event Segmentation.” Cognitive Science, 35(8), 1-29.

Mital, Parag K., Tim J. Smith, Robin Hill. and John M. Henderson. 2011. “Clustering of gaze during dynamic scene viewing is predicted by motion.” Cognitive Computation, 3(1), 5-24

Rayner, Keith. 1998. “Eye movements in reading and information processing: 20 years of research”. Psychological Bulletin, 124(3), 372-422.

Rayner, Keith, Tim J. Smith, George Malcolm and John M. Henderson, J.M. 2009. “Eye movements and visual encoding during scene perception.” Psychological Science, 20, 6-10.

Raz, Gal, Yael Jacob, Tal Gonen, Yonatan Winetraub, Tamar Flash, Eyal Soreq and Talma Hendler. 2014. “Cry for her or cry with her: context-dependent dissociation of two modes of cinematic empathy reflected in network cohesion dynamics.” Social cognitive and affective neuroscience, 9(1), 30-38.

Redmond, Sean, Jodi Sita and Kim Vincs. 2015. “Our Sherlockian Eyes: the Surveillance of Vision”, Refractory: a Journal of Entertainment Media, 25.

Robinson, Jennifer, Jane Stadler and Andrea Rassell. 2015. “Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye-tracking Lens”, Refractory: a Journal of Entertainment Media, 25.

Salt, Barry. 2009. Film Style and Technology: History and Analysis (Vol. 3rd). Totton, Hampshire, UK: Starword.

Sawahata, Yasuhito, Rajiv Khosla, Kazuteru Komine, Nobuyuki Hiruma, Takayuki Itou, Seiji Watanabe, Yuji Suzuki, Yumiko Hara and Nobuo Issiki. 2008. “Determining comprehension and quality of TV programs using eye-gaze tracking.” Pattern Recognition, 41(5), 1610-1626.

Smith, Murray. 2011. “Triangulating Aesthetic Experience”, paper presented at the annual Society for Cognitive Studies of the Moving Image conference, Budapest, June 8–11, 201

Smith, Tim J. 2006. An Attentional Theory of Continuity Editing. Ph.D., University of Edinburgh, Edinburgh, UK.

Smith, Tim J. 2012a. “The Attentional Theory of Cinematic Continuity”, Projections: The Journal for Movies and the Mind. 6(1), 1-27.

Smith, Tim J. 2012b. “Extending AToCC: a reply,” Projections: The Journal for Movies and the Mind. 6(1), 71-78

Smith, Tim J. 2013. “Watching you watch movies: Using eye tracking to inform cognitive film theory.” In A. P. Shimamura (Ed.), Psychocinematics: Exploring Cognition at the Movies. New York: Oxford University Press. pages 165-191

Smith, Tim J. 2014. “Audiovisual correspondences in Sergei Eisenstein’s Alexander Nevsky: a case study in viewer attention”. Cognitive Media Theory (AFI Film Reader), Eds. P. Taberham & T. Nannicelli.

Smith, Tim J., Jonathan Batten and Rachael Bedford. 2014. “Implicit detection of asynchronous audiovisual speech by eye movements.” Journal of Vision,14(10), 440-440.

Smith, Tim J., Dekker, T., Mital, Parag K., Saez De Urabain, I. R. & Karmiloff-Smith, A., In Prep. “Watch like mother: Motion and faces make infant gaze indistinguishable from adult gaze during Tot TV.”

Smith, Tim J. and John M. Henderson. 2008. “Edit Blindness: The relationship between attention and global change blindness in dynamic scenes”. Journal of Eye Movement Research, 2(2):6, 1-17.

Smith Tim J., Peter Lamont and John M. Henderson. 2012. “The penny drops: Change blindness at fixation.” Perception 41(4) 489 – 492

Smith, Tim J., Daniel Levin and James E. Cutting. 2012. “A Window on Reality: Perceiving Edited Moving Images.” Current Directions in Psychological Science. 21: 101-106

Smith, Tim J. and Parag K. Mital. 2013. “Attentional synchrony and the influence of viewing task on gaze behaviour in static and dynamic scenes”. Journal of Vision 13(8): 16.

Smith, Tim J. and Janet Y. Martin-Portugues Santacreu. Under Review. “Match-Action: The role of motion and audio in limiting awareness of global change blindness in film.”

Smith, Tim. J. and Murray Smith. In Prep. “The impact of expertise on eye movements during film viewing.”

Suckfull, Monika. 2000. “Film Analysis and Psychophysiology Effects of Moments of Impact and Protagonists”. Media Psychology2(3), 269-301.

Vilaro, Anna and Tim J. Smith. 2011. “Subtitle reading effects on visual and verbal information processing in films.” Published abstract In Perception. ECVP abstract supplement, 40. (p. 153). European Conference on Visual Perception. Toulousse, France.

Velichkovsky, Boris M., Sascha M. Dornhoefer, Sebastian Pannasch and Pieter J. A. Unema. 2001. “Visual fixations and level of attentional processing”. In Andrew T. Duhowski (Ed.), Proceedings of the International Conference Eye Tracking Research & Applications, Palm Beach Gardens, FL, November 6-8, ACM Press.

Wass, Sam V. and Tim J. Smith. In Press. “Visual motherese? Signal-to-noise ratios in toddler-directed television,” Developmental Science

Yarrow, Kielan, Patrick Haggard, Ron Heal, Peter Brown and John C. Rothwell. 2001. “Illusory perceptions of space and time preserve cross-saccadic perceptual continuity”. Nature, 414.

Zakay, Dan and Richard A. Block. 1996. Role of Attention in Time Estimation Processes. Time, Internal Clocks, and Movement. Elsevier Science.

 

Notes

[ii] An alternative take on eye tracking data is to divorce the data itself from psychological interpretation. Instead of viewing a gaze point as an index of where a viewer’s overt attention is focussed and a record of the visual input most likely to be encoded into the viewer’s long-term experience of the media, researchers can instead take a qualitative, or even aesthetic approach to the data. The gaze point becomes a trace of some aspect of the viewer’s engagement with the film. The patterns of gaze, its movements across the screen and the coordination/disagreement between viewers can inform qualitative interpretation without recourse to visual cognition. Such an approach is evident in several of the articles in this special issue (including Redmond, Sita, and Vincs, this issue; Batty, Perkins, and Sita, this issue). This approach can be interesting and important for stimulating hypotheses about how such patterns of viewing have come about and may be a satisfying endpoint for some disciplinary approaches to film. However, if researchers are interested in testing these hypotheses further empirical manipulation of the factors that are believed to be important and statistical testing would be required. During such investigation current theories about what eye movements are and how they relate to cognition must also be respected.

[iii] Although, one promising area of research is the use of pupil diameter changes as an index of arousal (Bradley, Miccoli, Escrig and Lang, 2008).

[iv] This technique has been used for decades by producers of TV advertisements and by some “pop” serials such as Hollyoaks in the UK (Thanks for Craig Batty for this observation).

[v] This trend in increasing pace and visual complexity of film is confirmed by statistical analyses of film corpora over time (Cutting, DeLong and Nothelfer, 2010) and has resulted in a backlash and increasing interest in “slow cinema”.

[vi] Other authors in this special issue may argue that taking a critical approach to gaze heatmaps without recourse to psychology allows them to embed eye tracking within their existing theoretical framework (such as hermeneutics). However, I would warn that eye tracking data is simply a record of how a relatively arbitrary piece of machinery (the eye tracking hardware) and associated software decided to represent the centre of a viewer’s gaze. There are numerous parameters that can be tweaked to massively alter how such gaze traces and heatmaps appear. Without understanding the psychology and the physiology of the human eye a researcher cannot know how to set these parameters, how much to trust the equipment they are using, or the data it is recording and as a consequence may over attribute interpretation to a representation that is not reliable.

[vii] https://theeyetribe.com/ (accessed 13/12/14). The EyeTribe tracker is $99 and is as spatially and temporally accurate (up to 60Hz sampling rate) as some science-grade trackers.

[viii] http://www.tobii.com/eye-experience/ (accessed 13/12/14). The Tobii EyeX tracker is $139, samples at 30Hz and is as spatially accurate as the EyeTribe although the EyeX does not give you as much access to the raw gaze data (e.g., pupil size and binocular gaze coordinates) as the EyeTribe.

 

Bio

Dr Tim J. Smith is a senior lecturer in the Department of Psychological Sciences at Birkbeck, University of London. He applies empirical Cognitive Psychology methods including eye tracking to questions of Film Cognition and has published extensively on the subject both in Psychology and Film journals.

 

Politicizing Eye tracking Studies of Film – William Brown

Abstract

This essay puts eye tracking studies of cinema into contact with film theory, or what I term film-philosophy, so as to distinguish film theory from specifically cognitive film theory. Looking at the concept of attention, the essay explains how winning and keeping viewers’ attention in a synchronous fashion is understood by eye tracking studies of cinema as key to success in filmmaking, while film-philosophy considers the winning and keeping of attention by cinema to be a political issue driven by economics and underscored by issues of control. As such, film-philosophy understands cinema as political, even if eye tracking studies of film tend to avoid engagement in political debate. Nonetheless, the essay identifies political dimensions in eye tracking film studies: the legitimization of the approach, its emphasis on mainstream cinema as an object of study and its emphasis on statistical significance all potentially have political connotations/ramifications. Invoking the concept of cinephilia, the essay then suggests that idiosyncratic viewer responses, as well as films that do not synchronously capture attention, might yield important results/play an important role in life in an attention-driven society.

In this essay, I wish to put eye tracking studies of film into dialogue with a more political approach to film, drawn from film theory, or what, for the benefit of distinguishing film theory from cognitive film theory, I shall term film-philosophy. In doing so, I shall draw out what for film-philosophy are some of the limitations of eye tracking, including its emphasis on statistical significance, or what most viewers look at when they watch films. I shall argue that we might learn as much, if not more, about cinema by paying attention not only to statistically significant and shared responses to films (what most viewers look at), but also to those viewers whose responses to a film do not form part of the statistically significant group, and/or to films that may not induce in viewers statistically significant and shared responses. In effect, we may find that there are insights to be derived from those who look at the margins of the cinematic image, rather than at the centre, even if those viewers are themselves ‘marginal’ in the sense that they are pushed to the margins of most/all eye tracking studies of film viewers. There is perhaps also value to be found in looking at ‘marginal’ films. In this way, we might find that idiosyncratic responses to a film or films is as important as the shared response. I shall also argue that there is a politics to the idiosyncratic response, especially when it is put into dialogue with film theoretical/film-philosophical work on cinephilia, and that as a result there is also a politics to eye tracking and its emphasis on statistical significance. I shall start, however, by looking at the state of eye tracking film research today.

On 29 and 30 July 2014, the Academy of Motion Picture Arts and Sciences (AMPAS) – the same American academy that distributes the so-called Oscars – held two events under the combined title of ‘Movies in Your Brain: The Science of Cinematic Perception’. The events included contributions from neuroscientists Uri Hasson, Talma Hendler and Jeffrey M. Zacks, psychologist James E. Cutting, directors Darren Aronofsky and Jon Favreau, editor Walter Murch and writer-producer Ari Handel. The host of the first evening was psychologist Tim J. Smith, whose eye tracking studies of cinema have arguably become the best known and most influential over recent years (see, inter alia, Smith 2012a; Smith 2013; Smith 2014). Through these events, as well as through coverage of these events in fashionable journals like Wired (Miller 2014a; Miller 2014b), we can see how eye tracking – together with the study of film using brain scanning technologies such as functional Magnetic Resonance Imaging (fMRI) – is clearly becoming important for our understanding of how films work. This in turn means that such studies are surely important to film studies.

For a detailed history and overview of eye tracking, explaining how it works and what it tells us about film, I cannot do better than to guide readers to the afore-mentioned work by Smith. Smith has soundly demonstrated, and with great clarity, how the human eye moves via small movements called saccades, and that in between saccades the human eye fixates. It is during fixations that humans take in visual information, with fixations being linked therefore to attention and to working memory; we tend to remember objects from our visual field upon which we have fixated, or to which we have paid attention. Clearly this is important to the study of film, since viewers typically attend only to parts of the movie screen at any given time, and not necessarily to others or to the whole of the screen (and the surrounding auditorium). Can/do filmmakers exert influence over where we look, for how long, and thus what we remember about a film – with those memories themselves lasting for greater or lesser periods of time? And if filmmakers do influence such things, how much influence do they exert and through which techniques? These are the questions that eye tracking technology can help to answer – and scholars like Smith do so with great skill and eloquence.

My aim, however, is not simply to reproduce findings by Smith and others who have used eye tracking devices to study film. In order to construct a theoretical argument concerning the importance of the idiosyncratic, or ‘cinephilic’, response to a film or films in general, as well as the importance of a filmmaker not necessarily ‘controlling’ where a viewer looks, but instead allowing/encouraging viewers precisely to look idiosyncratically, cinephilically, or where they wish, I need instead to bring the scientific and ‘apolitical’ use of eye tracking devices into a political discourse concerning the nature of cinema, power, hegemony and the issue of cinematic homogeneity and/or heterogeneity. This is a controversial maneuver – in that it will bring together two areas of film studies that often seem to stand in ‘opposition’ to each other, namely cognitive film theory and a film theory that still plies its trade using Continental philosophy, or what for the sake of simplicity I shall term film-philosophy. My desire is not simply to be controversial, however. Rather it is to engage with what eye tracking means to film studies, both currently and potentially in the future.

To begin to bring eye tracking studies of film into the ‘political discourse’ mentioned above, I shall relate an anecdote. A semi-regular response from colleagues in film studies, when I tell them about eye tracking studies of film viewing, is that eye tracking doesn’t tell us anything about films that we didn’t already know. Is it a surprise that we tend to look more often at the center of the screen? Is it a surprise that we typically attend more to brightly illuminated parts of the screen than to dimly lit ones? Is it a surprise that we tend to direct our attention towards human faces when watching a film that features human characters? Anyone who has consciously thought about what they do while watching a film will be able to tell from memory alone that these things are all true. As a result, eye tracking studies of film can sometimes be filled with what, at least to the film student/scholar, are truisms. By way of an example, Paul Marchant and colleagues say that ‘these strategies and techniques… [capture] the audience’s visual attention: focus, camera movement, eye line match, color and contrast, motion of elements within the shot, graphic matching’ (Marchant et al. 2009, 158). On my print-out of Marchant et al.’s essay, my own apostil next to this assertion reads as follows: ‘Do we not know this already (otherwise cinema would not have developed these techniques)?’ Many, if not all, film viewers will know simply from experience that these techniques help to guide their attention, even if they are blissfully unaware of the relationship between eye fixations, attention and memory. Of course, it is pleasing to have our introspective responses to/our intuitive knowledge about cinema ‘scientifically’ confirmed (to a large extent, but not entirely – about which, more later); but essentially, so my colleagues’ argument goes, eye tracking studies tell us what we already know.

Now, even if I myself find some eye tracking studies of film to be ‘truistic’, I nonetheless believe that eye tracking studies of film are of great importance. However, their importance is perhaps in playing a role that is different from the one that eye tracking studies of film seem to give to themselves, which is as a key component of cognitive film theory. Instead, I think that eye tracking studies of film are important for film theory, or what today is termed film-philosophy. I shall explain the distinction between cognitive film theory and film-philosophy presently.

Little in this world is uniform, and so by definition I generalize when I say that the basic tenet of cognitive film theory is, with David Bordwell and Noël Carroll’s Post Theory: Reconstructing Film Studies (1996) serving as its figurehead, for film studies to move towards a theory of cinema based on the analysis of films themselves, and away from a film theory that uses cinema as a means of confirming or denying a Lacanian understanding of the human and/or an Althusserian/Marxist conception of contemporary capital. In spite of cognitive film theory’s lack of uniformity, eye tracking studies of film are nonetheless part of cognitive film theory’s project to help us to look at cinema ‘as it is’, and not to use cinema as a political football. Conversely, film-philosophy is in general informed by the kinds of Continental philosophers, often though not limited to Gilles Deleuze, that cognitive film theorists reject, and it engages not just with films ‘as they are’, but with the politics of films.

Now, to claim that we can isolate films and film viewing from a human world that is perhaps always political, and to claim that we can then analyse films ‘as they are’, is perhaps absurd: films ‘as they are’ are part of a political world, and cognitive film theorists are not unaware of this, just as film-philosophers are not incapable of scientific analysis. However, how much politics is allowed into the analysis of films perhaps informs the broad distinction between cognitive film theory and film-philosophy, as I hope to clarify by looking briefly at the role of attention in the work of two scholars, Tim J. Smith and Jonathan Beller. In his ‘Attentional Theory of Cinematic Continuity’ (AToCC), Smith (2012a) uses eye tracking studies to demonstrate how filmmakers capture and maintain viewers’ attention, with certain techniques, mainly those associated with continuity editing, being more successful than others. Meanwhile, in his Cinematic Mode of Production: Attention Economy and the Society of the Spectacle, Beller (2006) suggests that capturing attention is not necessarily an aesthetic, but rather a political project: the more attention a film garners, the more success one will have in monetizing that film, with the making of money becoming the bottom line of cinema. Beller does not appeal to some early cinema that did not attempt to elicit viewers’ attention and thus make money; such an early cinema did not necessarily exist. Rather, Beller argues that cinema has always been part of an economy that is based on attention; indeed, cinema plays a key role in naturalizing this attention economy, meaning that cinema has not always been necessarily capitalist, but that the capitalist world endeavors as much as possible to become cinematic, to capture our attention as much as possible in order to ‘win’ the economic race, since capturing eyeballs means making money. Smith explains how attention is captured; Beller offers an explanation as to why. Even though filmmakers rely on natural processes in order to capture attention (Smith), the process of consistently trying to capture our attention (‘cinema’) is not natural, but political and economic (Beller).

James E. Cutting, in commenting on an earlier draft of this paper, says that the results of eye tracking studies of film, which reveal how filmmakers capture attention, are

big news… because almost nothing else does this – not static pictures (photographs, artworks), not class room behavior by teachers, not leaders of business meetings, and often not even spectacles of various kinds (sporting events, rock concerts, etc); even TV is typically not as good as the average narrative, popular movie. (Cutting, signed peer review 2014)

If cinema is indeed better at capturing our attention than these other media, and if in some senses it is better at capturing our attention than those parts of the world that do not feature such media – i.e. if cinema is better at capturing our attention than reality – then cinema, and the making-cinematic of reality in a bid to capture attention, to make money and/or to influence people (Cutting compares cinema in particular to teachers and to business leaders) is profoundly political. It is profoundly political because learning about how to capture attention – learning about how cinema works – is tied to the shaping of our material reality (putting screens everywhere) and to controlling attention (encouraging us to look at those screens, and not at the rest of reality). Cognitive film theory is apolitical; film-philosophy, meanwhile, engages in the very political dimensions of cinema. Eye tracking studies of film tend to position themselves as part of the former; my aim here is to bring them into dialogue with the latter.

If eye tracking studies of film tend to position themselves as part of a would-be apolitical approach to cinema, then in their investigation into cinema, they are nonetheless conducting an investigation into politics, as per Beller’s equation of cinema with politics highlighted above. However, while eye tracking studies of film position themselves as apolitical, politics do creep into eye tracking studies, especially through what I shall call their absences. What is more, these politics do relate to film-philosophy’s ‘political’ approach to film. In order to demonstrate this, I shall begin by analyzing how eye tracking studies of film have sought historically to legitimate themselves.

Early in an essay that gives an overview of eye tracking studies of film, Smith asserts, without naming any, that the hypotheses of film theory ‘generally remain untested’ (Smith 2013, 165). In this almost throwaway comment, we perhaps find important information. For in asserting that eye tracking is what can help us to ‘test’ out some theories of film, as Smith goes on to do in relation to Sergei M. Eisenstein’s writing about his own film, Alexander Nevsky (USSR, 1938), he perhaps overlooks how film theorists often (but perhaps not always) try (though not always with success) to construct their theories based on the films that they have seen, studied and perhaps made, and not the other way around. That is, Smith seems not to consider that watching films is itself a means of testing our theories about films – without the need for eye tracking devices. On a related note, while he does consider filmmakers like Eisenstein, D.W. Griffith, Edward Dmytryk and others as ‘experimentalists’ of sorts (who have tested their own theories), Smith also does not fully acknowledge that the history of cinema can itself be seen as a prolonged ‘test’ in what ‘works’ or ‘does not work’ with audiences – with that which ‘works’ being regularly adopted as either a short- or a long-term strategy by the film industry, be that in terms of re-using storylines, adopting a specific cinematic style, employing bankable film stars, using topical settings, engaging with zeitgeist themes and so on. Instead, it is Smith’s intervention that will validate or otherwise that history of theory and practice, and which will confirm what filmmakers, and perhaps also many audience members, have probably known for a long time, even if putting their knowledge into practice sometimes proves harder than we might imagine (because otherwise films would presumably not have ‘mistakes’ in them).

Now, it’s natural that a (relatively) new approach to studying film would need to legitimize itself in order to gain credibility and following – and Smith clearly charts the c30 year trajectory of eye tracking in film studies since the 1980s onwards (Smith 2014: 90). Nonetheless, if the history of cinema is not ‘test’ enough for Smith, then implicitly a claim is being made here about what constitutes a ‘real’ test, and, by extension, what sort of person can carry out a ‘real’ test. In other words, eye tracking, and the cognitive framework more generally, here legitimizes itself as being a tool for verifying (scientifically) what previously were ‘mere’ and speculative theories (these are my terms) – with the people qualified to carry out these tests being neither filmmakers nor audience members, but psychologists. By justifying eye tracking in this way, Smith is not just making a statement of fact (eye tracking demonstrates that viewers look at the same things at the same time during films made using the continuity editing style), but he is also – I assume unintentionally – making an implicit value judgment that carries political assumptions regarding what constitutes a/the most legitimate framework for learning and knowing about film. If, as per my anecdote above, I can and do know the same things via introspection that eye tracking tells me, then why is introspection not equally legitimate as a framework, even if the former involves less visible labor, and certainly less sexy imagery, and thus does not seem to involve any real ‘testing’?

Eye tracking thus seeks ‘politically’ to legitimate itself as a tool for film analysis. To be clear: eye tracking is legitimate, but it is also always already making claims about what constitutes knowledge: introspection is not knowledge, while science is – even if both can lead to the same understanding. Importantly, in producing visible evidence (the afore-mentioned ‘sexy imagery’ of colored clusters of eye-gaze on scenes from films), then eye tracking studies are also always already cinematic, by which I mean to say that they affirm a system whereby the visual/the cinematic (here are pictures of attention being captured) are validated above invisible (here, introspective) approaches to the same knowledge. This in turn always already affirms the process of cinema and attention-grabbing as being the (political) system that is most powerful.

If eye tracking affirms a politically cinematic world, in that cinematic forms of knowledge are more valid than invisible, i.e. uncinematic, ones, then within that cinematic world eye tracking might also, and in some respects implicitly does, legitimate some forms of cinema over others. This is suggested by the way in which eye tracking studies look predominantly at Hollywood/mainstream cinema in their analyses of film. For example, in his AToCC, Smith (2012a) cites a diverse range of movies, including L’escamotage d’une dame au théâtre Robert Houdin/The Vanishing Lady (Georges Méliès, France, 1896) and L’année dernière à Marienbad/Last Year at Marienbad (Alain Resnais, France/Italy, 1961), but eye tracking data are given mainly for contemporary Hollywood films, including Blade Runner (Ridley Scott, USA/Hong Kong/UK, 1982), Requiem for a Dream (Darren Aronofsky, USA, 2000) and There Will Be Blood (Paul Thomas Anderson, USA, 2007), with Smith suggesting that continuity editing is the form of cinema best suited to capturing attention.1

The absence of eye tracking data on those other, non-Hollywood films is perhaps telling, as suggested by two respondents to Smith’s essay, who query how his theories would apply to different cinemas, including the avant garde (Freeland 2012, 40-41) and, at least by implication, Japanese cinema (Rogers 2012, 47-48). Eye tracking would of course yield important insights into avant-garde and other forms of cinema, but that information is not offered here.

Furthermore, Smith’s suggestion that continuity editing is the form best suited to capturing attention, also prompts Paul Messaris and Greg M. Smith to argue that continuity editing violations, in particular jump cuts, are quite regular and not particularly detrimental to the continuity of the film viewing experience (Messaris 2012, 28-29; Greg M. Smith 2012, 57). Malcolm Turvey, meanwhile, argues that the film viewing experience is always continuous, meaning that the ‘continuity’ of continuity editing ‘is not continuity of viewer attention per se… but rather the manner in which films engage and manage that attention’ (Turvey 2012, 52-53; for Smith’s riposte to these responses and more, see Smith 2012b).

These responses highlight how filmmaking ‘perfection’ (an absence of continuity errors) need not be fetishized too much; audiences are quite happy to watch films with continuity errors (many of which they will not notice). Furthermore, many audiences love what Jeffrey Sconce (1995) might term ‘paracinema’ – i.e. ‘trash’ cinema and ‘bad’ movies – be they intentionally ‘bad’ or otherwise. In other words, it would seem that as long as audiences are primed regarding how they should receive a film (or, in Turvey’s language, as long as their attention is managed and then engaged in the right way), then you don’t need to care about and can even love the stylised acting, the ropey mise-en-scène, the unmotivated camera movements, the strange edits and the story loopholes of, say, The Room (Tommy Wiseau, USA, 2003), supposedly the worst film in history. Under the right circumstances (with the right management/ preparation), it would seem that audiences can like pretty much anything, including a 485-minute film of the Empire State Building (Empire, Andy Warhol, USA, 1964). In other words, while in his AToCC Smith mentions Méliès and Resnais, and while he engages with Eisenstein and other filmmakers elsewhere, the AToCC puts an emphasis on mainstream Hollywood cinema and its predominant system of continuity editing, since this cinema elicits a synchronicity of response, or control over attention, in that viewers attend to the same parts of the screen at the same time – while also often failing to detect edits done in the continuity editing style (see Smith and Henderson 2008). There is a seeming bias here towards mainstream, narrative filmmaking, the engrossing nature of which is lauded at the expense of other cinemas.

Let us move away from Smith in order to demonstrate how this bias is not his alone. Jennifer Treuting suggests that ‘[t]he use of eye tracking… can help filmmakers and other visual artists refine their craft’ (Treuting 2006: 31). In some respects, this is an innocent comment; I have no doubt that eye tracking can help filmmakers and other visual artists to refine their craft. But suggested in this ‘refinement’ is also the move towards validating the mainstream/continuity style at the expense of its alternatives. A combined eye tracking and fMRI study carried out by Uri Hasson and colleagues also makes this clear: much fuss is made over how work by Alfred Hitchcock elicits greater synchrony (‘inter-subject correlation’) in viewers than does an ‘unstructured’ shot of a concert in Washington Square Park, a film that is simply a ‘point of reference’ and which ‘fails to direct viewers’ gaze’ (Hasson et al. 2008, 13-14; emphasis added). My reference above to Warhol’s Empire here becomes apposite: what Hasson and colleagues dismiss as a ‘point of reference’ and as a ‘failure’ in various respects defines one of the great experimental films. Perhaps ‘marginal’ films like Empire should also be considered successful – but at achieving something different to the work of Hitchcock, and perhaps Hasson’s film is not a ‘point of reference’, but an experimental work that equally inhabits the totality of films in the world that we shall call cinema.

If Hitchcock ‘succeeds’ in controlling viewers’ attention, while Warhol by implication ‘fails’, then eye tracking becomes implicitly/inevitably embroiled in not just what film is, but in what film could or should be – as Treuting’s suggestion that eye tracking might feed back into filmmaking also makes clear. This suggests that there is a politics to eye tracking film studies, particularly in the UK where universities are increasingly relying on ‘impact’, particularly on the economy, in order to survive: they don’t just observe films, but feed back into how films are, or should be, made, by exploring what is ‘successful’ in terms of eliciting attention, getting bums on seats and thus making money. In some respects, eye tracking in particular and cognitive film theory in general are now dragged back towards the Marxist approach to cinema that cognitive film theory initially sought to reject: it, too, shapes/seeks to shape cinema just as Marxist film theory in effect lobbied for alternatives to the mainstream. However, where Marxist film theory lobbied for a rejection of mainstream cinematic techniques, eye tracking studies seem to validate them – and to suggest that filmmakers might ‘refine their craft’ by adopting/intensifying them. Saving the thorny issue of ‘control’ and ‘influence’ for later, there is still a political dimension to this potential validation of mainstream cinema techniques, because it reaffirms the economic hegemony of one style over others and it also validates in some degree a homogeneity of product (and of audience?) – all within a ‘cinematic’ economic system that is itself predicated upon gaining attention. Cinema is both business and art, but if art is one thing it is unique/different, and so a move towards homogeneity is a move towards the reduction of art in favor of business. If it requires an artist rather than an academic to make this clear, then Darren Aronofsky’s apprehensive response to Hasson’s work at the AMPAS events hopefully serves this purpose: ‘“It’s a scary tool for the studios to have,” Aronofsky said. “Soon they’ll do test screenings with people in MRIs.” The audience laughed, but it didn’t seem like he was joking, at least not entirely’ (Miller 2014b).

I have so far argued that cinema is political, that eye tracking studies have required some political maneuvering in order to legitimate themselves, and that the focus on continuity editing/mainstream cinema by eye tracking studies may also have a political dimension. However, are eye tracking studies themselves without methodological politics, in that they simply report findings? I wish presently to suggest that eye tracking research does have methodological limitations – which is why I asserted above that eye tracking film studies are only to a large extent and not entirely reliable – and that these limitations also have a political dimension. The methodological limitations are not simply a case of potential inaccuracies regarding the type of eye-tracker used, determining how long the eye needs to be still for a fixation to take place, what algorithm is used to measure this, or how accurate is the eye-tracker in determining where exactly the eye is looking – all ongoing issues with eye tracking technologies (see, inter alia, Wass et al. 2013; Saez de Urabain et al. 2014). It is also a case of issues of statistical significance and the politics thereof, particularly what I shall call the temporal politics, and to a lesser extent the social politics, of eye tracking. In relation to the latter, many eye tracking studies involve students in order to carry out their research (e.g. Tatler et al. 2010; Võ et al. 2012). As a result, the findings might pertain not universally, but to population members who are of a certain age and, if we can say that university students tend to be from more affluent backgrounds, a certain socioeconomic status. In relation to statistical significance, meanwhile, all studies tend to discount those viewers who do not look where the researchers want them to look; for example, in a study of where people look when viewing moving faces, only 87 per cent of fixations targeted the face region when shown a moving face with sound, with that figure dropping to 82 per cent when shown a moving face without sound (Võ et al. 2012, 7). Of course, when what one is investigating is where people look when they look at faces, it is correct to discount those 13-18 per cent of fixations that were not directed at the face. But the point is that similar discounts happen all the time, not least in the process of averaging that we see in various experiments, including those mentioned by Marchant et al., Hasson et al., and Smith. And yet, where neuroscience is based in large part upon the study of anomalous brains – from autists to damage sufferers to perceived geniuses – psychologists engaged in eye tracking tend to go with force majeure and report the average, or what most people do. There may however be in human populations a ‘long tail’ (to use the terminology of Chris Anderson, 2006) that may not in any one experiment be statistically significant, but which over a number of experiments might begin to show patterns that could help us to understand vision and attention in a more ‘holistic’ fashion.

To continue by way of another anecdote: a film scholar took part in an eye tracking film study at a leading European university. Upon completion, the colleague conducting the study told the scholar that they looked in completely different places – generally at the margins of the screen – to where most of the other participants looked, and that their participation was therefore useless to the study. If we can say that the film scholar looked (perhaps deliberately) where others do not look, then to what degree is film viewing a matter of, to use Turvey’s language, management and engagement? That is, do film scholars look differently at films, perhaps even at the world? And if so, what can we make of this?

The Russian ‘godfather’ of eye tracking studies, Alfred Yarbus. famously published in the 1960s that setting viewers different tasks will completely modify where they look at an image (Yarbus 1967; see also Tatler et al. 2010). There is much to extrapolate from this. For while eye tracking studies will use terms like ‘naïve’ to define how participants are unaware of the aims of the study, when it comes to film viewing, humans are rarely naïve at all. Advertising, reviews and other publicity materials are always – at least on an implicit level – telling us how and where to look at films, just as the media and our conspecifics are telling us how and where to look in the real world. Now, it may well be that humans who have never before seen a movie have little trouble understanding Hollywood cinema, as affirmed, inter alia, by both Messaris (2012, 31-33) and Smith (2012b, 74). Nonetheless, our attention is not just managed and engaged in the cinema, but it is also managed and engaged for the cinema, and I have not read any studies where psychologists showed a non-Hollywood film to first-time audiences and in which those audiences had trouble understanding the film; that is, these studies affirm nothing about the comprehension of continuity editing per se, although they might affirm that humans can understand cinema without training – as is presumably affirmed worldwide everyday as the first film shown to children is not a Hollywood film but a Bollywood, Nollywood, Filipino, Chinese or other movie; what is more, the studies perhaps only affirm the cultural hegemony enjoyed by Hollywood, in that psychologists present a Hollywood and not another film to those first-time viewers – and then use that research to affirm Hollywood’s economic primacy as being a result of its filmmaking style and not also as a result of historical and other factors. As Cynthia Freeland reminds us in her response to Smith’s AToCC, James Peterson in Post Theory argued that

a common feature of avant-garde film viewing – one that usually passes without comment: viewers initially have difficulty comprehending avant-garde films, but they learn to make sense of them. Students who take my course in the avant-garde cinema are at first completely confused by the films I show; by the end of term, they can speak intelligently about the films they see. (Peterson 1996, 110; quoted in Freeland 2012, 41)

In other words, as per my assertions re: The Room above, it is quite possible that humans would quite easily watch – and enjoy – all manner of different films, but that they do not because their attention is not ‘managed and engaged’. Again, this is a political issue, because if it is true, then it is about who can afford to use the mass media to manage and engage the attention of the most people in the quest for profit – meaning that alternative approaches to filmmaking are forced either to adopt the same system of filmmaking to compete, or they are pushed to the margins where the struggle to find audiences – because people are not prepped to watch them. The scholar at the European university has had a long education in film, and this potentially helps to manage and engage differently how they attend to them; their ‘statistically insignificant’ response might well be important in helping to demonstrate how we can not just view different/marginal films, but also view mainstream films differently.

Cutting and colleagues suggest that film editing correlates with a 1/f pattern, with 1/f (one over frequency) referring to the ‘natural’ amount of time that humans attend to objects in the real world (Cutting et al. 2010). In other words, the suggestion is that Hollywood editing rhythms reflect human attention spans – ‘evolving toward 1/f spectra… [meaning that] the mind can be “lost”… most easily in a temporal art form with that structure’ (Cutting et al. 2010, 7). Now, since David L Gilden only came up with the 1/f structure in 1995 (Gilden et al. 1995), it remains untested, and untestable without a time machine, as to whether the human attention span itself changes over time, or according to culture. That said, if cinema has always been going at about the pace that human attention was working, and if cinema cutting rates have accelerated since the 1930s and through to the present era, then attention spans may well interact with culture, and even be shaped by our media.

I often ask my students how long they should look at a painting for. It’s a trick question, because of course there is no right or wrong answer. It is my (untested) hypothesis, however, that the amount of time humans look at paintings has been shaped by the media, including films; that is, in galleries, I see people look at paintings for about the average duration of a film shot (four to five seconds) – although recently they have begun to look at a painting for about the amount of time that it takes them to take a photo of that painting with their mobile handheld device.2 Smith, citing Cutting’s work, suggests that

[i]n an average movie theatre with a 40-foot screen viewed at a distance of 35 feet, this region at the centre of our gaze will only cover about 0.19 per cent of the total screen area. Given that the average shot length of most films produced today is less than 4 seconds… viewers will only be able to make at most 20 fixations covering only 3.8 per cent of the screen area. (Smith 2013: 168)

Given that paintings vary in size, one cannot rightly say how long it would take to see a ‘whole’ painting. But if one looks at a cinema-screen sized painting for 4 seconds, then one would, after Smith, fixate on about 4 per cent of that painting. In order to see the whole painting, then more time is needed, just as more time is needed to take in our natural, rather than cinematic, environment, since we also only ever see a small proportion of that at any one time.

Relating to film the foregoing foray into painting, we might add that, given that we do not take in visual information while saccading, and given that saccades have a duration of 20-50 miliseconds (Smith 2013, 168), this means that we do not take in visual information for 0.7 seconds during every four-second shot. At 90 minutes in length, there are on average 1,350 shots per film, meaning that we do not take in visual information for 15 minutes and 45 seconds per film – blinks and turning away from the screen for snogging and toilet breaks not included. If spatially we only see 3.8 per cent of the screen during a shot, and if we only see 82.5 per cent of a film’s duration, this means that we see around 3.14 per cent of the average (Hollywood) film (no spooky π references intended).3 To be clear, these statistics apply not just to Hollywood: I would only see 3.14 per cent of Empire if I were to watch it at the cinema, too. But since it is a film comprised of a single-seeming shot and a static frame, Empire clearly encourages viewers to look for longer at the space within the frame, while Hollywood arguably does not give viewers the time to do so, since the content and duration of images is concerned uniquely with story-telling, and not with anything else. This in turn affects for how long we think that we are supposed to look at objects in our everyday lives, if for the sake of argument my gallery hypothesis be allowed to stand. Neither paintings, nor Empire, nor the world itself is organized to be seen ‘cinematically’, even if Empire is undoubtedly a work of cinema. That is, they all invite contemplation, but what they often receive is a shot-length of attention before they become boring (Empire perhaps deliberately so). Neither paintings, nor Empire, nor large swathes of the world itself controls our attention in the way cinema does; there would be much more idiosyncrasy and less synchrony of attention when looking at Empire than at a mainstream film. If the proliferation of screens featuring cinematic techniques is the making-cinematic of reality in the services of capital, then the refusal to attend to paintings, Empire and the world itself suggests not just that our attention is controlled while watching a film, but that our attention is working at a ‘cinematic’ rhythm – a rhythm that Empire uses the very apparatus of cinema in order to try to break.

The ‘temporal politics’ that I mentioned above, then, is to do with the management and engagement of attention rhythms/patterns not just in cinema, and not just for cinema (we are prepped to be movie viewers), but also by cinema for the world (people pay attention to paintings in galleries about as long as they would attend to a film shot/as long as a film shot would allow them to attend to it, before ‘cutting’, or turning away, likely getting out one’s phone, the screen of which one can also cut across with the swipe of a thumb). Politics rear their head again as homogeneity of attention span, perhaps even of life rhythm, jump into bed with the political and economic concerns that govern the structures of our society. Almost certainly in an unwitting fashion (this is not a conspiracy), validating certain cutting rates and attentions spans over others becomes an issue linked to social control, and the economic bottom line of both cinema and perhaps society as a whole. Eye tracking studies of film are part of this political ecology.

A final throw of the dice. Those of us engaged in education are of course part of a system that prepares our students for the real world. But I am personally also committed to encouraging my students sociably and communicatively to develop their individuality, to become ‘idiosyncratic’, to look at the world differently and various other notions that have long since been corporatized disingenuously as advertising slogans. Being a film teacher, I do this through encouraging my students to look differently at films. Hollywood films employ techniques that do not encourage us to look differently at movies; instead, our attention (and our brain activity) are synchronized. What is more, the idiosyncratic viewers that do look at films differently (the European film scholar) are discounted from eye tracking studies for not conforming to the norm (for not confirming to us what we already know, even if not through a scientific framework). Not only might we encourage our students to look at the world differently (to become the idiosyncratic, perhaps ‘educated’ viewer), but we might also encourage our students to make films differently, since films can also play a role in encouraging us to see the world differently, to become ‘idiosyncratic’ individuals (Hasson’s research involved the production of an interesting avant garde work, regardless of his own thoughts on the matter). Perhaps eye tracking (and fMRI) studies can help in this by turning their attention not to the majority, but to the minority, to the marginal people who look, both figuratively and literally, at the margins of the screen, and at marginal films. And this perhaps involves slowing attention down, and making it (willfully?) deeper rather than rapid and superficial. I know that the longer I look at a painting, the more the power of its creation comes to my mind, the more I marvel at it and also at the world that sustains it. In other words, it brings me joy. As I repeat often to those students who do not seem committed to participating in my classes: the more you put in, the more you get out.

Would to educate (to manage and engage attention) both in the classroom and through making and showing different sorts of (slower?) films not simply replace one trend with another, and itself be prey to political issues regarding what type of ‘idiosyncrasy’ is best? Of course, such questions are going to be of ongoing importance and would need constant attention. In relation to eye tracking film studies, though, the introduction of a ‘temporal’ dimension might help enrich our understanding of idiosyncrasy. The spatial information that idiosyncratic eye-tracks give to us is chaotic and without pattern – and thus of not much use to the psychologist; however, there may well be temporal patterns that emerge when we consider ‘idiosyncrasy’ as a shared process (to be encouraged?), rather than as a reified thing to be commoditized.

Paul Willemen has written about cinephilia as being the search for/paying attention to otherwise overlooked details in movies (Willemen 1994, 223-57). Meanwhile, Laura Mulvey has argued that DVD technology allows the film viewer to develop a deeper, cinephilic relationship with movies, since she can now pause and really analyse a film – by ‘delaying’ it/slowing it down (Mulvey 2006, 144-60). To look idiosyncratically at a movie is thus to look ‘cinephilically’; it is to look at cinema with love, perhaps to look with love tout court – but in this instance at cinema. My argument comes full circle, then, as we bring cognitive film theory, via eye tracking film studies, into contact with film theory/film-philosophy, exemplified here by Mulvey as a major figure from the Screen movement/moment. There is no I in eye tracking – but if we can accept that eye tracking studies of cinema are embroiled in a political discourse (and a political reality) concerning which films are validated as better than others and why, then perhaps by putting an ‘I’ into eye tracking, by looking at the idiosyncratic in addition to the statistically significant, then we may be able to bring about different ways of seeing and making films.

 Notes

  1. The exception is Dancer in the Dark (Lars von Trier, Spain/Argentina/ Denmark/Germany/Netherlands/Italy/USA/UK/France/Sweden/Finland/ Iceland/Norway, 2000).
  2. One of my peer reviewers took issue with the speculative nature of this suggestion. The other agreed with it.
  3. Note that I insist on the term ‘visual information’ – since film does not just engage us visually, but also aurally and via other senses (as Freeland, 2012, also reminds Smith in her response to his AToCC essay).

 

References

Anderson, Chris. 2006. The Long Tail: Why the Future of Business is Selling Less of More. New York: Hyperion.

Beller, Jonathan. 2006. The Cinematic Mode of Production: Attention Economy and the Society of the Spectacle. Lebanon, N.H.: Dartmouth College Press.

Bordwell, David. 2010. “Now you see it, now you can’t.” Observations on Film Art: Kristin Thompson and David Bordwell, June 21. http://www.davidbordwell.net/blog/?p=8509.

Bordwell, David, and Noël Carroll. 1996. Post-Theory: Reconstructing Film Studies. Madison: University of Wisconsin Press.

Cutting, James E.. 2014. Peer Reviewer’s Comments. Received October 1.

Cutting, James E., Jordan E. DeLong and Christine E. Nothelfer. 2010. “Attention and the Evolution of Hollywood Film.” Psychological Science 20:10, 1-8.

Freeland, Cynthia. 2012. “Continuity, Narrative, and Cross-Modal Cuing of Attention.” Projections: The Journal for Movies and Mind 6:1, 34-42.

Gilden, D.L., T. Thornton, and M.W. Mallon. 1995. “1/f Noise in Human Cognition.” Science 267:1837-39.

Hasson, Uri, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin and David J. Heeger. 2008. “Neurocinematics: The Neuroscience of Film.” Projections: The Journal for Movies and Mind 2:1, 1-26.

Marchant, Paul, David Raybould, Tony Renshaw and Richard Stevens. 2009. “Are you seeing what I’m seeing? An eye tracking evaluation of dynamic scenes.” Digital Creativity 20:3, 153-163.

Messaris, Paul. 2012. “Continuity and Its Discontents.” Projections: The Journal for Movies and Mind 6:1, 28-33.

Miller, Greg. 2014a. “How Movies Manipulate Your Brain to Keep You Entertained.Wired, August 26.

Miller, Greg. 2014b. “How Movies Synchronize the Brains of an Audience.” Wired, August 28.

Mulvey, Laura. 2006. Death 24x a Second: Stillness and the Moving Image. London: Reaktion.

Peterson, James. 1996. “Is a Cognitive Approach to the Avant-Garde Cinema Perverse?” In Post Theory: Reconstructing Film Studies, edited by David Bordwell and Noël Carroll, 108-129. Madison: University of Wisconsin Press.

Rogers, Sheena. 2012. “Auteur of Attention: The Filmmaker as a Cognitive Scientist.” Projections: The Journal for Movies and Mind 6:1, 42-49.

Saez de Urabaín, Irati R., Mark H. Johnson and Tim J. Smith. 2014. “GraFIX: A semiautomatic approach for parsing low- and high-quality eye tracking data.” Behavior Research Methods, March 27, pp. 1-20.

Sconce, Jeffrey. 1995. “‘Trashing’ the academy: taste, excess, and an emerging politics of cinematic style.” Screen 36:4, 371-393.

Smith, Greg M. 2012. “Continuity Is Not Continuous.” Projections: The Journal for Movies and Mind 6:1, 56-61.

Smith, Tim J. 2012a. “The Attentional Theory of Continuity Editing.” Projections: The Journal for Movies and Mind 6:1, 1-27.

Smith, Tim J. 2012b. “Extending AToCC: A Reply.” Projections: The Journal for Movies and Mind 6:1, 71-78.

Smith, Tim J. 2013. “Watching You Watch Movies: Using Eye Tracking to Inform Cognitive Film Theory.” In Psychocinematics: Exploring Cognition at the Movies, edited by Art P. Shimamura, 165-191. New York: Oxford University Press.

Smith, Tim J. 2014. “Audiovisual Correspondences in Sergei Eisenstein’s Alexander Nevsky: A Case Study in Viewer Attention.” In Cognitive Media Theory, edited by Ted Nannicelli and Paul Taberham, 85-105. London: Routledge/American Film Institute.

Smith, Tim J, and John M. Henderson. 2008. “Edit Blindness: The relationship between attention and global change blindness in dynamic scenes.” Journal of Eye Movement Research 2(2):6, 1-17.

Tatler, Benjamin W., Nicholas J. Wade, Hoi Kwan, John M. Findlay and Boris M. Velichkovsky. 2010. “Yarbus, eye movements, and vision.” i-Perception 1:7-27.

Treuting, Jennifer. 2006. “Eye Tracking and the Cinema: A Study of Film Theory and Visual Perception.” SMPTE Motion Imaging Journal 115:1, 31-40.

Turvey, Malcolm. 2012. “The Continuity of Narrative Comprehension.” Projections: The Journal for Movies and Mind 6:1, 49-56.

Võ, Melissa L.-H., Tim J. Smith, Parag K. Mital and John M. Henderson. 2012. “Do the eyes really have it? Dynamic allocation og attention when viewing moving faces.” Journal of Vision 12(13):3, 1-14.

Wass, Sam V., Tim J. Smith and Mark H. Johnson. 2013. “Parsing eye tracking data of variable quality to provide accurate fixation duration estimates in infants and adults.” Behavior Research Methods 45:1, 229-250.

Willemen, Paul. 1994. Looks and Frictions: Essays in Cultural Studies and Film Theory. Bloomington: Indiana University Press.

Yarbus, Alfred L. 1967. Eye Movements and Vision. Translated by Basil Haigh. New York: Plenum Press.

 

William Brown is a Senior Lecturer in Film at the University of Roehampton, London. He is the author of Supercinema: Film-Philosophy for the Digital Age (Berghahn, 2013) and, with Dina Iordanova and Leshu Torchin, of Moving People, Moving Images: Cinema and Trafficking in the New Europe (St Andrews Film Studies, 2010). He is the co-editor, with David Martin-Jones, of Deleuze and Film (Edinburgh University Press, 2012). He is also a filmmaker.

Subtitles on the Moving Image: an Overview of Eye Tracking Studies – Jan Louis Kruger, Agnieszka Szarkowska and Izabela Krejtz

Abstract

This article provides an overview of eye tracking studies on subtitling (also known as captioning), and makes recommendations for future cognitive research in the field of audiovisual translation (AVT). We find that most studies in the field that have been conducted to date fail to address the actual processing of verbal information contained in subtitles, and rather focus on the impact of subtitles on viewing behaviour. We also show how eye tracking can be utilised to measure not only the reading of subtitles, but also the impact of stylistic elements such as language usage and technical issues such as the presence of subtitles during shot changes on the cognitive processing of the audiovisual text as a whole. We support our overview with empirical evidence from various eye tracking studies conducted on a number of languages, language combinations, viewing contexts as well as different types of viewers/readers, such as hearing, hard of hearing and Deaf people.

Introduction

The reading of printed text has received substantial attention from scholars since the 1970s (for an overview of the first two decades see Rayner et al. 1998). Many of these studies, conducted from a psycholinguistic angle, made use of eye tracking. As a result, a large body of knowledge exists on the eye movements during reading of people with varying levels of reading skills and language proficiency, with a range of ages, different first languages and cultural backgrounds, and in different contexts. Studies on subtitle reading, however, have not achieved the same level of scientific rigour largely for practical reasons: subtitles are not static for more than a few seconds at a time; they compete for visual attention with a moving image; and they compete for overall cognitive resources with verbal and non-verbal sounds. This article will identify some of the gaps in current research in the field, and also illustrate how some of these gaps can be bridged.

Studying the reading of subtitles is significantly different from studying the reading of static text. In the first place, as far as eye tracking software is concerned, the subtitles appear on a moving image as image rather than text, which renders traditional text-based reading statistics and software all but useless. This also makes the collection of data for reading research on subtitles a painstakingly slow process involving substantial manual inspections and coding. Secondly, the fact that subtitles appear against the background of the moving image means that they are always in competition with this image, which renders the reading process fundamentally different from the reading process of static texts: on the one hand because the reading of subtitles compete with the processing of the image, sometimes resulting in interrupted reading, but on the other hand the limited time the subtitles are on screen means that readers have less time to reread or regress to study difficult words or to check information. Either way, studying this reading process, and the cognitive processing that takes place during the reading, is much more complicated than in the case of static texts where we know that the reader is mainly focussing on the words before her/him without additional auditory and visual information to process.

While the viewing of subtitles has been the object of many eye tracking studies in recent years, with increasing frequency (see, for example Bisson et al. 2012; d’Ydewalle and Gielen 1992; d’Ydewalle and De Bruycker 2007; Ghia 2012; Krejtz et al. 2013; Kruger 2013; Kruger et al. 2013; Kruger and Steyn 2014; Perego et al. 2010; Rajendran et al. 2013; Specker 2008; Szarkowska et al. 2011; Winke et al. 2013), the study of the reading of subtitles remains a largely uncharted territory with many research avenues still to be explored. Those studies that do venture to measure more than just attention to the subtitle area, seldom do this for extended texts.

In this article we provide an overview of studies on how subtitles change the way viewers process audiovisual material, and also of studies on the unique characteristics of the subtitle reading process. Taking an analysis of the differences between reading printed (static) text and subtitles as point of departure, we examine a number of aspects typical of the way subtitle text is processed in reading. We also look at the impact of the dynamic nature of the text and the competition with other sources of information on the reading process (including scene perception, changes in the viewing process, shifts between subtitles and image, visual saliency of text, faces, and movement, and cognitive load), as well as discussing studies on the impact of graphic elements on subtitle reading (e.g. number of lines, and text chunking), and studies that attempt to measure the subtitle reading process in more detail.

We start off with a discussion of the way in which watching an audiovisual text with subtitles alters viewing behaviour as well as of the complexities of studying subtitles due to the dynamic nature of the image it has as a backdrop. Here we focus on the fleeting nature of the subtitle text, the competition between reading the subtitles and scanning the image, and the interaction between different sources of information. We further discuss internal factors that impact on subtitle processing, like the language and culture of the audience, the language of the subtitles, the degree of access the audience has to sound, and other internal factors, before turning to external factors related to the nature of the audiovisual text and the presentation of the subtitles. Finally, we provide an overview of studies attempting to measure the processing of subtitles as well as findings from two studies that approach the processing of subtitles

The dynamic nature of the subtitle reading process

Reading subtitles differs substantially from reading printed text in a number of respects. As opposed to “static text on a stable background”, the viewer of subtitled audiovisual material is confronted with “fleeting text on a dynamic background” (Kruger and Steyn 2014, 105). In consequence, viewers not only need to process and integrate information from different communication channels (verbal visual, non-verbal visual, verbal auditory, non-verbal auditory, see Gottlieb 1998), but they also have no control over the presentation speed (see Kruger and Steyn 2014; Szarkowska et al. forthcoming). As a consequence, unlike in the reading of static texts, the pace of reading is in part dictated by the text rather than the reader – by the time the text is available to be read – and there is much less time for the reader to regress to an earlier part of a sentence or phrase, and no opportunity to return to previous sentences. Reading traditionally takes place in a limited window which the reader is acutely aware will disappear in a few seconds. Even though there are exceptions to the level of control a viewer has, for example in the case of DVD and PVR as well as other electronic media where the viewer can rewind and forward at will, the typical viewing of subtitles for most audiovisual products happens continuously and without pauses just as when watching live television.

Regressions, which form an important consideration in the reading of static text, take on a different aspect in the context of the knowledge (the viewer has) that dwelling too much on any part of a subtitle may make it difficult to finish reading the subtitle before it disappears. Any subtitle is on screen for between one and six seconds, and the viewer also has to simultaneously process all the other auditory (in the case of hearing audiences) and visual cues. In other words, unlike when reading printed text, reading becomes only one of the cognitive processes the viewer has to juggle in order to understand the audiovisual text as a whole. Some regressions are in fact triggered by the change of the image in shot changes (and to a much lesser extent scene changes) when the text stays on across these boundaries, which means that the viewer sometimes returns to the beginning of the subtitle to check whether it is a new subtitle, and sometimes even re-reads the subtitle. For example, in a recent study, Krejtz et al. (2013) established that participants tend not to re-read subtitles after a shot change or cut. But their data also revealed that a proportion of the participants did return their gaze to the beginning of the subtitle after such a change (see also De Linde and Kay, 1999). What this means for the study of subtitle reading is that these momentary returns (even if only for checking) result in a class of regressions that is not in fact a regression to re-read a word or section, but rather a false initiation of reading for what some viewers initially perceive to be a new sentence.

On the positive side, the fact that subtitles are embedded on a moving image and are accompanied by a soundtrack (in the case of hearing audiences) facilitates the processing of language in context. Unfortunately, this context also introduces competition for attention and cognitive resources. For the Deaf and hard of hearing audience, attention has to be divided between reading the subtitles and processing the scene, extracting information from facial expressions, lip movements and gestures, and matching or checking this against the information obtained in the subtitles. For the hearing audience who makes use of subtitles for support or to provide access to foreign language dialogue, attention is likewise divided between subtitles and the visual scene, and just as the Deaf and hard of hearing audiences have the added demand on their cognitive resources of having to match what they read with what they get from non-verbal signs and lip movements, the hearing audience matches what they read with what they hear, checking for correspondence of information and interpreting intonation, tenor and other non-verbal elements of speech.

What stands beyond doubt is that the appearance of subtitles changes the viewing process. In 2000, Jensema et al. famously stated that “the addition of captions to a video resulted in major changes in eye movement patterns, with the viewing process becoming primarily a reading process” (2000a, 275). Having examined the eye movements of six subjects watching video clips with and without subtitles, they found that the onset of a subtitle triggers a change in the eye movement pattern: when a subtitle appears, viewers move their gaze from whatever they were watching in order to follow the subtitle. In a more wide-scale study it was concluded by d’Ydewalle and de Bruycker (2007,196) that “paying attention to the subtitle at its presentation onset is more or less obligatory and is unaffected by major contextual factors such as the availability of the soundtrack, knowledge of the foreign language in the soundtrack, and important episodic characteristics of actions in the movie: Switching attention from the visual image to “reading” the subtitles happens effortlessly and almost automatically”.

Subtitles therefore appear to be the cause of eye movement bias similar to faces (see Hershler & Hochstein, 2005; Langton, Law, Burton, & Schweinberger, 2008; Yarbus, 1967), the centre of the screen, contrast and movement. In other words, subtitles attract the gaze at least in part because of the fact that the eye is drawn to the words on screen just as the eye is drawn to movement and other elements. Eyes are drawn to subtitles not only because the text is identified as a source of meaningful information (in other words a top-down impulse as the viewer consciously consults the subtitles to obtain relevant information), but also because of the change to the scene that the appearance of a subtitle causes (in other words a bottom-up impulse, automatically drawing the eyes to what has changed on the screen).

As in most other contexts, the degree to which viewers will process the subtitles (i.e. read them rather than merely look at them when they appear and then look away) will be determined by the extent to which they need the subtitles to follow the dialogue or to obtain information on relevant sounds. In studying visual attention to subtitles it therefore remains a priority to measure the degree of processing, something that has not been done in more than a handful of studies, and something to which we will return later in the article.

Viewers usually attend to the image on the screen, but when subtitles appear, it only takes a few frames for most viewers to move their gaze to read the subtitles. The fact that people tend to move their gaze to subtitles the moment they appear on the screen is illustrated in Figures 1 and 2.

Figure. 1 Heat maps of three consecutive film stills – Polish news programme Fakty (TVN) with intralingual subtitles.

Figure. 1 Heat maps of three consecutive film stills – Polish news programme Fakty (TVN) with intralingual subtitles.

Figure 2. Heat maps of two consecutive film stills – Polish news programme Wiadomości (TVP1) with intralingual subtitles

Figure 2. Heat maps of two consecutive film stills – Polish news programme Wiadomości (TVP1) with intralingual subtitles

Likewise, when the gaze of a group of viewers watching an audiovisual text without subtitles is compared to that of a similar group watching the same text with subtitles, the split in attention is immediately visible as the second group reads the subtitles and attends less to the image, as can be seen in Figure 3.

Figure 3. Heat maps of the same scene seen without subtitles and with subtitles – recording of an academic lecture.

Figure 3. Heat maps of the same scene seen without subtitles and with subtitles – recording of an academic lecture.

Viewer-internal factors that impact on subtitle processing

The degree to which the subtitles are processed is far from straightforward. In a study performed at a South African university in the context of Sesotho students looking at a recorded lecture with subtitles in their first language and audio in English (their language of instruction), students were found to avoid looking at the subtitles (see Kruger, Hefer and Matthew, 2013b). Sesotho students in a different group who saw the same lecture with English subtitles processed the subtitles to a much larger extent. This contrast is illustrated in the focus maps in Figures 4.

4-Kruger

Figure 4. Focus maps of Sesotho students looking at a lecture in intralingual English subtitles (left) and another group looking at the same lecture with interlingual Sesotho subtitles (right) – recording of an academic lecture.

The difference in eye movement behaviour between the conditions is also evident when considering the number of subtitles skipped. Participants in the above study who saw the video with Sesotho subtitles skipped an average of around 50% of the Sesotho subtitles (median at around 58%), whereas participants who saw the video with English subtitles only skipped an average of around 20% of the English subtitles (with a median of around 8%) (see Kruger, Hefer & Matthew, 2014).

This example does not, however, represent the conventional use of subtitles where viewers would rely on the subtitles to gain access to a text from which they would have been excluded without the subtitles. It does serve to illustrate that subtitle reading is not unproblematic and that more research is needed on the nature of processing in different contexts by different audiences. For example, in a study in Poland, interlingual subtitles (English to Polish) were skipped slightly less often by hearing viewers compared to intralingual subtitles (Polish to Polish), possibly because hearing viewers didn’t need them to follow the plot (see Szarkowska et al., forthcoming).

Another important finding from eye tracking studies on the subtitle process relates to how viewers typically go about reading a subtitle. Jensema et al. (2000) found that in subtitled videos, “there appears to be a general tendency to start by looking at the middle of the screen and then moving the gaze to the beginning of a caption within a fraction of a second. Viewers read the caption and then glance at the video action after they finish reading” (2000, 284). This pattern is indeed often found, as illustrated in the sequence of frames from a short video from our study in Figure 5.

Figure 5. Sequence of typical subtitle reading – a recording of Polish news programme Fakty (TVN) with intralingual subtitles.

Figure 5. Sequence of typical subtitle reading – a recording of Polish news programme Fakty (TVN) with intralingual subtitles.

Some viewers, however, do not read so smoothly and tend to shift their gaze between the image and the subtitles, as demonstrated in Figure 6. The gaze shifts between the image and the subtitle, also referred to in literature as ‘deflections’ (de Linde and Kay 1999) or ‘back-and-forth shifts’ (d’Ydewalle and De Bruycker (2007), can be regarded as an indication of the smoothness of the subtitle reading process: the fewer the gaze shifts, the more fluent the reading and vice versa.

Figure 6. Scanpath of frequent gaze shifting between text and image – a recording of Polish news programme Fakty (TVN) with intralingual subtitles.

Figure 6. Scanpath of frequent gaze shifting between text and image – a recording of Polish news programme Fakty (TVN) with intralingual subtitles.

An important factor that influences subtitle reading patterns is the nature of the audience. In Figure 7 an interesting difference is shown between the way a Deaf and a hard of hearing viewer watched a subtitled video. The Deaf viewer moved her gaze from the centre of the screen to read the subtitle and then, after having read the subtitle, returned the gaze to the centre of the screen. In contrast, the hard of hearing viewer made constant comparisons between the subtitles and the image, possibly relying on residual hearing and trying to support the subtitle reading process with lip-reading. Such a result was reported by Szarkowska et al. (2011), who found differences in the number of gaze shifts between the subtitles and the image in the verbatim subtitles condition, particularly discernible (and statistically significant) in the hard of hearing group (when compared to the hearing and Deaf groups).

Figure 7. Scanpaths of Deaf and hard of hearing viewers. Left: Gaze plot illustrating the viewing pattern of a Deaf participant watching a clip with verbatim subtitles.  Right: Gaze plot illustrating the viewing pattern of a hard of hearing participant watching a clip with verbatim subtitles.

Figure 7. Scanpaths of Deaf and hard of hearing viewers. Left: Gaze plot illustrating the viewing pattern of a Deaf participant watching a clip with verbatim subtitles. Right: Gaze plot illustrating the viewing pattern of a hard of hearing participant watching a clip with verbatim subtitles.

These provisional qualitative indications of differences between eye movements of users with different profiles require more in-depth quantitative investigation and the subsequent section will provide a few steps in this direction.

As mentioned above, subtitle reading patterns largely depend on the type of viewers. Fluent readers have been found to have no difficulty following subtitles. Diao et al. (2007), for example, found a direct correlation between the impact of subtitles on learning and the academic and literacy levels of participants. Similarly, given that “hearing status and literacy tend to covary” (Burnham et al. 2008, 392), some previous studies found important differences in the way hearing and hearing-impaired people watch subtitled programmes. Robson (2004, 21) notes that “regardless of their intelligence, if English is their second language (after sign language), they [i.e. Deaf people] cannot be expected to have the same comprehension levels as hearing people who grew up exposed to English”. This is indeed confirmed by Szarkowska et al. (forthcoming) who report that Deaf and hard of hearing viewers in their study made more fixations on subtitles and that their dwell time on the subtitles was longer compared to hearing viewers. This result may indicate a larger effort needed to process subtitled content and more difficulty in extracting information (see Holmqvist et al. 2011, 387-388). This, in turn, may stem from the fact that for some Deaf people the language in the subtitles is not their mother tongue (their L1 being sign language). At the same time, for hearing-impaired viewers, subtitles provide an important source of information on the words spoken in the audiovisual text as well as other information contained in the audio track, which in itself explains the fact that they would spend more time looking at the subtitles.

Viewer-external factors that impact on subtitle processing

The ‘smoothness’ of the subtitle reading process depends on a number of factors, including the nature of the audiovisual material as well as technical and graphical aspects of subtitles themselves. At a general level, genre has an impact on both the role of subtitles in the total viewing experience, and on the way viewers process the subtitles. For example, d’Ydewalle and Van Rensbergen (1989) found that children in Grade 2 paid less attention to subtitles if a film involved a lot of action (see d’Ydewalle & Bruycker 2007 for a discussion). The reasons for this could simply be that action film tends to have less dialogue in the first place, but secondly and more significantly, the pace of the visual editing and the use of special effects creates a stronger visual element which shifts the balance of content towards the action (visual content) and away from dialogue (soundtrack and therefore subtitles). This, however, is an area that has to be investigated empirically. At a more specific level, technical characteristics of an audiovisual text such as film editing have an impact on the processing of subtitles.

1 Film editing

Film editing has a strong influence on the way people read subtitles, even beyond the difference in editing pace as a result of genre (for example, action and experimental films could typically be said to have a higher editing pace than dramas and documentaries). In terms of audience perception, viewers have been found to be unaware of standard film editing techniques (such as continuity editing) and are thus able to perceive film as a continuous whole in spite of numerous cuts – the phenomenon termed “edit blindness” (Smith & Henderson, 2008, 2). With more erratic and fast-paced editing, it stands to reason that the cognitive demands will increase as viewers have to work harder to sustain the illusion of a continuous whole.

When subtitles clash with editing such as cuts (i.e. if subtitles stay on screen over a shot or scene change), conventional wisdom as passed on by generations of subtitling guides (see Díaz Cintas & Remael 2007, ITC Guidance on Standards for Subtitling 1999) suggests that the viewer will assume that the subtitle has changed with the image and as a consequence they will re-read it (see above). However, Krejtz et al. (2013) reported that subtitles displayed over shot changes are more likely to cause perceptual confusion by making viewers shift their gaze between the subtitle and the rest of the image more frequently than subtitles which do not cross film cuts (cf. de Linde and Kay 1999). As such, the cognitive load is bound to increase.

2 Text chunking and line segmentation

Another conventional wisdom, perpetuated in subtitling guidelines and standards, is that poor line segmentation will result in less efficient processing (see Díaz Cintas & Remael 2007, Karamitroglou 1998). In other words, subtitles should be chunked per line and between subtitles in terms of self-contained semantic units. The line of dialogue: “He told me that he would meet me at the red mailbox” should therefore be segmented in something like the following ways:

He told me he would meet me
at the red mailbox.

Or

He told me
he would meet me at the red mailbox.

Neither of the following segmentations would be optimal because the prepositional phrase ‘at the red mailbox’ and the verb phrase ‘he would meet me’, respectively, are split, which is considered an error:

He told me he would meet me at the
red mailbox

He told me he
would meet me at the red mailbox.

However, Perego et al. (2010) found that poor line segmentation in two-line subtitles did not affect subtitle comprehension negatively. They also investigated 28 subtitles viewed by 16 participants using a threshold line between the subtitle region and the upper part of the screen, or main film zone, but did not find a statistically significant difference between the well-segmented and ill-segmented subtitles in terms of fixation counts, total fixation time, or number of shifts between subtitle region and upper area. The only statistically significant difference they found was between the mean fixation duration within the subtitle area between the two conditions, with the mean fixation duration in the ill-segmented subtitles being on average 12ms longer than in the well-segmented subtitles. Although the authors downplay the importance of this difference on the grounds that the difference is so small, it does seem to indicate at least a slightly higher cognitive load when the subtitles are ill-segmented. The small number of subtitles and participants, however, make it difficult to generalize from their results, again a result of the fact that it is difficult to extract reading statistics for subtitles unless the reading behaviour can be quantified over longer audiovisual texts.

In a study conducted a few years later, Rajendran et al. (2013) found that “chunking improves the viewing experience by reducing the amount of time spent on reading subtitles” (2013, 5). This study compared conditions different from those investigated in the previous study, excluding the ill-segmented condition of Perego et al. (2010), and focused mostly on live subtitling with respeaking. In the earlier study, which focused on pre-recorded subtitling, the subtitles in the two conditions were essentially still part of one sense unit that appeared as one two-line subtitle. In the later study, the conditions were chunked by phrase (similar to the well-segmented condition of the earlier study but with phrases appearing one by one on one line), no segmentation (where the subtitle area was filled with as much text as possible with no attempt at segmentation), word by word (where words appeared one by one) and chunked by sentence (where the sentences showed up one by one). Regardless of the fact that this later study therefore essentially investigated different conditions, they did find that the most disruptive condition was where the subtitle appeared word by word – eliciting more gaze points (defined less strictly than in fixation algorithms used by commercial eye trackers) and more “saccadic crossovers” or switches between image and subtitle area. However, in this study by Rajendran et al. (2013), the videos were extremely short (under a minute), and the sound was muted, hampering the ecological validity of the material, and once again making the findings less suitable to generalization.

Although both these studies have limitations in terms of generalizability, they both provide some indication that segmentation has an impact on subtitle processing. Future studies will nonetheless have to investigate this aspect over longer videos to determine whether the graphical appearance, and particularly the segmentation of subtitles, has a detrimental effect on subtitle processing in terms of cognitive load and effectiveness.

3 Language

The language of subtitles has received considerable attention from psycholinguists in the context of subtitle reading. D’Ydewalle and de Bruycker (2007) examined eye movement behaviour of people reading standard interlingual subtitles (with the audio track in a foreign language and subtitles in their native language) and reversed subtitles (with the audio in their mother tongue and subtitles in a foreign language). They found more regular reading patterns in the standard interlingual subtitling condition, with the reversed subtitling condition having more subtitles skipped, fewer fixations per subtitle, etc. (see also d’Ydewalle and de Bruycker 2003 and Pavakanun 1993). This is an interesting finding in itself, as it is the reversed subtitling that has been found to be particularly conducive to foreign language learning (see Díaz Cintas and Fernández Cruz 2008, and Vanderplank 1988).

Szarkowska et al. (forthcoming) examined differences in reading patterns of intralingual (Polish to Polish) and interlingual (English to Polish) subtitles among a group of Deaf, hard of hearing and hearing viewers. They found no differences in reading for the Deaf and hard of hearing audiences, but hearing people made significantly more fixations to subtitles when watching English clips with interlingual Polish subtitles than Polish clips with intralingual Polish subtitles. This confirms that the hearing viewers processed the subtitles to a significantly lower degree when they were redundant, as in the case of intralingual transcriptions of the soundtrack. What would be interesting to investigate in this context is those instances when the hearing audience did in fact read the subtitles, to determine to what extent and under what circumstances the redundant written information is used by viewers to support their auditory intake of information.

In a study on the influence of translation strategies on subtitle reading, Ghia (2012) investigated the differences in the processing of literal vs. non-literal translations into Italian of an English film clip (6 minutes) when watched by Italian EFL learners. According to Ghia, just as subtitle format, layout, and segmentation have the potential to affect visual and perceptual dynamics, the relationship translation establishes with the original text means that “subtitle translation is also likely to influence the perception of the audiovisual product and viewers’ general reading patterns” (2012,175). Ghia particularly wanted to investigate the processing of different translation strategies in the presence of sound and image with the subtitles. In her study she found that the non-literal translations (where the target text diverged from the source text) resulted in more deflections between text and image. This is similar to the findings of Rajendran et al. (2013) in terms of less fluent graphics in word-by-word subtitles.

As can be seen from the above, the aspect of language processing in the context of subtitled audiovisual texts has received some attention, but has not to date been approached in any comprehensive manner. In particular, there is a need for more psycholinguistic studies to determine how subtitle reading differs from the reading of static text, and how this knowledge can be applied to the practice of subtitling.

Measuring subtitle processing

1 Attention distribution and presentation speed

In the study by Jensema et al. (2000), subjects spent on average 84% of the time looking at subtitles, 14% at the video picture and 2% outside of the frame. The study represents an important early attempt to identify reading patterns in subtitle reading, but it has considerable limitations. The study had only six participants, three deaf and three hearing, and the video clips were extremely short (around 11 seconds each), presented with English subtitles (in upper case) without sound. The fact that there was no soundtrack therefore impacted on the time spent on the subtitles. In Perego et al’s study (2010), the ratio is reported as 67% on the subtitle area and 33% on the image. In this study there were 41 Italian participants who watched a 15-minute clip with Hungarian soundtrack and subtitles in Italian. As in the previous study, the audience therefore had to rely heavily on the subtitles in order to follow the dialogue. Kruger et al. (2014), in the context of intralingual subtitles in a Psychology lecture in English, found a ratio of 43% on subtitles, 43% on the speaker and slides and 14% on the rest of the screen. When the same lecture was subtitled into Sotho, the ratio changed to 20% on the subtitles, 66% on the speaker and slides, and 14% on the rest of the screen. This wide range is an indication of the difference in the distribution of visual attention in different contexts with different language combinations, different levels of redundancy of information, and differences in audiences.

In order to account for “the audiovisual nature of subtitled programmes”, Romero-Fresco (in press) puts forward the notion of ‘viewing speed’ – as opposed to reading speed and subtitling speed – which he defines as “the speed at which a given viewer watches a piece of audiovisual material, which in the case of subtitling includes accessing the subtitle, the accompanying images and the sound, if available”. The perception of subtitled programmes is therefore a result of not only the subtitle reading patterns, but also the visual elements of the film. Based on the analysis of over seventy-one thousand subtitles created in the course of the Digital Television for All project, Romero Fresco provides the following data on the viewing speed, reflecting the proportion of time spent by viewers looking at subtitles and at the images, proportional to the subtitle presentation rates (see Table 1).

Viewing speed Time on subtitles Time on images
120wpm ±40% ±60%
150wpm ±50% ±50%
180wpm ±60%-70% ±40%-30%
200wpm ±80% ±20%

Table 1. Viewing speed and distribution of gaze between subtitles and images (Romero-Fresco) 

Jensema et al. also suggested that the subtitle presentation rate may have an influence on the time spent reading subtitles vs. watching the rest of the image: “higher captioning speed results in more time spent reading captions on a video segment” (2000, 275). This was later confirmed by Szarkowska et al. (2011), who found that viewers spent more time on verbatim subtitles displayed at higher presentation rates compared to edited subtitles displayed with low reading speed, as illustrated by Figure 8.

Figure 8. Fixation-count based heatmaps illustrating changes in attention allocation of hearing and Deaf viewers watching videos subtitled at different rates.

Figure 8. Fixation-count based heatmaps illustrating changes in attention allocation of hearing and Deaf viewers watching videos subtitled at different rates.

2 Mean fixation duration

Irwin (2004, 94) states that “fixation location corresponds to the spatial locus of cognitive processing and that fixation or gaze duration corresponds to the duration of cognitive processing of the material located at fixation”. Within the same activity (e.g. reading), longer mean fixation durations could therefore be said to reflect more cognitive processing and higher cognitive load. One would therefore expect viewers to have longer fixations when the subject matter is more difficult, or when the language is more specialized. Across activities, however, comparisons of fixation duration is less meaningful as reading elicits more shorter fixations than scene perception or visual scanning simply because of the nature of the activities. It is therefore essential in eye tracking studies of subtitle reading to distinguish between the actual subtitles when they are on screen, the rest of the screen, and the subtitle area when there is no text (between successive subtitles).

The difference between reading and scene perception is illustrated in Figure 9, demonstrating that fixations on the image tend to be longer (indicated here by a bigger circle) than those on subtitles (which indicates more focused viewing), and more exploratory in nature (see the distinction between focal and ambient fixations in Velichkovsky et al. 2005).

Figure 9. Differences in fixation durations between the image and subtitle text – from Polish TV series Londyńczycy.

Figure 9. Differences in fixation durations between the image and subtitle text – from Polish TV series Londyńczycy.

Rayner (1984) indicated the impact of different tasks on mean fixation durations, as reflected in Table 2 below:

Task Mean fixation duration (ms) Mean saccade size (degrees)
Silent reading 225 2 (about 8 letters)
Oral reading 275 1.5 (about 6 letters)
Visual search 275 3
Scene perception 330 4
Music reading 375 1
Typing 400 1 (about 4 letters)

 Table 2. Approximate Mean Fixation Duration and Saccade Length in Reading, Visual Search, Scene Perception, Music Reading, and Typing[1]

In subtitling, silent reading is accompanied by simultaneous processing of the same information in the soundtrack (in the same or another language) as well as of other sounds and visual signs (for a hearing audience, that is – for a Deaf audience, it would be text and visual signs). The difference in mean fixation duration in these different tasks therefore reflects the difference in cognitive load. In silent reading of static text, there is no external competition for cognitive resources. When reading out loud, the speaker/reader inevitably monitor his/her own reading, introducing additional cognitive load. As the nature of the sign becomes more abstract, the load, and the fixation duration increases, and in the case of typing, different processing, production and checking activities are performed simultaneously, resulting in even higher cognitive load. This is inevitably an oversimplification of cognitive load, and indeed the nature of information acquisition between reading successive groups of letters (words) in a linear fashion is significantly different from that of scanning a visual scene for cues.

Undoubtedly, subtitle reading imposes different cognitive demands, and these demands are also very much dependent on the audience. In an extensive study on the differences in subtitle reading between Deaf, hard of hearing and hearing participants, we found a high degree of variation in mean fixation duration between the groups, and also a difference between the mean fixation duration in the Deaf and the hard of hearing groups between subtitles presented at 12 characters per second and 15 characters per second (see Szarkowska et al. forthcoming).

  12 characters per second 15 characters per second
Deaf 241.93 ms 232.82 ms
Hard of hearing 218.51 ms 214.78 ms
Hearing 186.66 ms 186.58 ms

Table 3. Differences in reading subtitles presented at different rates

Statistical analyses performed on the three groups with mean fixation duration as a dependent variable and groups and speed as categorical factors produced a statistically significant main effect, further confirmed by subsequent t-tests that yielded statistically significant differences in mean fixation duration for both subtitling speeds between all three groups. The difference within the Deaf and hard of hearing groups was also significant between 12cps and 15cps. What this suggests is that reading speed has a more pronounced effect on Deaf and hard of hearing viewers than on hearing ones.

3 Subtitle reading

As indicated at the outset, one of the biggest hurdles in studying the processing of subtitles is the fact that the subtitles appear as image on image rather than text on image as far as eye tracking analysis software is concerned. Whereas reading statistics software can therefore automatically mark words as areas of interest in static texts, and then calculate number of regressions, refixations, saccade length, fixation duration and count as related to the specific words, this process has to be done manually for subtitles. The fact that it is virtually impossible to create similar areas of interest on the subtitle words that are embedded in the image over large numbers of subtitles makes it very difficult to obtain reliable eye tracking results on subtitles as text. This explains the predominance of measures such as fixation count and fixation duration as well as shifts between subtitle area and image in eye tracking studies on subtitle processing. As a result, many of these studies do not distinguish directly between looking at the subtitle area and reading the subtitles, and, “they tend to define crude areas of interest (AOIs), such as the entire subtitle area, which means that eye movement data are also collected for the subtitle area when there are no subtitles on screen, which further skews the data” (Kruger and Steyn, 2014, 109).

Although a handful of studies come closer to studying subtitle reading by going beyond the study of fixation counts, mean fixation duration, and shifts between subtitle area and image area, most studies tend to focus on amount of attention rather than nature of attention. Briefly, the exceptions can be identified in the following studies: Specker (2008) looks at consecutive fixations; Perego et al. (2010) add the path length (sum of saccade lengths in pixels) to the more conventional measures; Rajendran et al. (2013) add the proportion of gaze points; Ghia (2012) looks at fixations on specific words as well as regressions; Bisson et al. (2012) look at the number of subtitles skipped, and proportion of successive fixations (number of successive fixations divided by total number of fixations); and in one of the most comprehensive studies on the subject of subtitle processing, d’Ydewalle and De Bruycker (2007) look at attention allocation (percentage of skipped subtitles, latency time, and percentage of time spent in the subtitle area), fixations (number, duration, and word-fixation probability), and saccades (saccade amplitude, percentage of regressive eye movements, and number of back-and-forth shifts between visual image and subtitle).

In a recent study, Kruger and Steyn (2014) provide a reading index for dynamic texts (RIDT) designed specifically to measure the degree of reading that takes place when subtitled material is viewed. This index is explained as “a product of the number of unique fixations per standard word in any given subtitle by each individual viewer and the average forward saccade length of the viewer on this subtitle per length of the standard word in the text as a whole” (2014, 110). Taking the location and start time of successive fixations within the subtitle area when a subtitle is present as the point of departure, the number of unique fixations (i.e. excluding refixations, and fixations following a regression) is determined, as well as the average length of forward saccades in the subtitle. This information gives an indication of the meaningful processing of the words in the subtitle when the number of fixations per word, as well as the length of saccades as ratio of the length of the average word in the audiovisual text are calculated. Essentially, the formula quantifies the reading of a particular subtitle by a particular participant by measuring the eye movement during subtitle reading against what is known about eye movements during reading and perceptual span.

In a little more detail, the formula can be written as follows for video v, with participant p viewing subtitle s”:

10

(Kruger and Steyn, 2014, 110).

This index was validated by performing a comparison of the manual inspection of the reading of 145 subtitles by 17 participants, and makes it possible to study the reading of subtitles over extended texts. In their study, Kruger and Steyn (2014) use the index to determine the relationship between subtitle reading and performance in an academic context, finding a significant positive correlation between the degree to which participants read the subtitles and their performance in a test written after watching subtitled lectures. The RIDT therefore presents a robust index of the degree to which subtitles are processed over extended texts, and could add significant value to psycholinguistic studies on subtitles. Using the index, previous claims that subtitles have a positive or negative impact on comprehension, vocabulary acquisition, language learning or other dependent variables, can be correlated with whether or not viewers actually read the subtitles, and to what extent the subtitles were read.

Conclusion

From this overview of studies investigating the processing of subtitles on the moving image it should be clear that much still needs to be done to gain a better understanding of the impact of various independent variables on subtitle processing. The complexity of the multimodal text, and in particular the competition between different sources of information, means that a subtitled audiovisual text is a substantially altered product from a cognitive perspective. Much progress has been made in coming to grips with the way different viewers behave when looking at subtitled audiovisual texts, but there are still more questions than answers – relating, for instance, to differences in how people process subtitled content on various devices (cf. the HBBTV4ALL project). The use of physiological measures like eye tracking and EEG (see Kruger et al. 2014) in combination with subjective measures like post-report questionnaires is, however, continually bringing us closer to understanding the impact of audiovisual translation like subtitling on the experience and processing of audiovisual texts.

 

Acknowledgements

This study was partially supported by research grant No. IP2011 053471 “Subtitling for the deaf and hard of hearing on digital television” from the Polish Ministry of Science and Higher Education for the years 2011–2014.

 

References

Bisson, Marie-Josée, Walter Van Heuven, Kathy Conklin, and Richard Tunney. 2014. “Processing of Native and Foreign Language Subtitles in Films: An Eye Tracking Study.” Applied Psycholinguistics 35(2):399-418.

Burnham, Denis, Leigh Greg, Noble William, Jones Caroline, Tyler Michael, Grebennikov Leonid and Alex Varley. 2008. Parameters in television captioning for deaf and hard-of-hearing adults: effects of caption rate versus text reduction on comprehension. Journal of Deaf Studies and Deaf Education 13 (3):391-404.

de Linde, Zoé and Neil Kay. 1999. The Semiotics of Subtitling. Manchester: St. Jerome.

Diao, Y., Chandler, P., Sweller, J. 2007. The effect of written text on comprehension of spoken English as a foreign language. The American Journal of Psychology 120(2): 237-261.

Díaz Cintas, Jorge and Marco Fernandez Cruz. (2008) “Using subtitled video materials for foreign language instruction”. In The Didactics of Audiovisual Translation edited by Jorge Díaz Cintas, 201-214. Amsterdam/Philadelphia: John Benjamins.

Díaz Cintas, Jorge and Aline Remael. 2007. Audiovisual Translation: Subtitling. Manchester: St. Jerome.

d’Ydewalle, Géry and Wim De Bruycker. 2003. Reading native and foreign language television subtitles in children and adults. In The mind’s eyes: Cognitive and applied aspects of eye movement research, edited by J. Hyönä, R. Radach and H. Deubel, 444-461. New York: Springer-Verlag.

d’Ydewalle, Géry and Wim De Bruycker. 2007. “Eye Movements of Children and Adults while Reading Television Subtitles.” European Psychologist 12:196–205.

d’Ydewalle, Géry and Ingrid Gielen. 1992. “Attention Allocation with Overlapping Sound, Image, and Text.” In Eye Movements and Visual Cognition: Scene Perception and Reading, edited by Keith Rayner, 415–427. New York: Springer-Verlag.

d’Ydewalle, Géry, Johan Van Rensbergen, and Joris Pollet. 1987. Reading a message when the same message is available auditorily in another language: The case of subtitling. In Eye Movements: From Physiology to Cognition edited by J.K O’Reagan and A. Lévy Schoen, 313-321. Amsterdam: Elsevier Science Publishers B.V. (North-Holland).

Ghia, Elisa. 2012. “The Impact of Translation Strategies on Subtitle Reading.” In Eye Tracking in Audiovisual Translation, edited by Elisa Perego, 155–182. Roma: Aracne Editrice.

Gottlieb, Henrik. 1998. Subtitling. In Routledge Encyclopaedia of Translation Studies, edited by Mona Baker, 244-248. London & New York: Routledge.

Hershler, Orit and Shaul Hochstein. 2005. At first sight: a high-level pop out effect for faces. Vision Research, 45, 1707–1724.

Holmqvist, Kenneth et al. 2011. Eyetracking. A Comprehensive Guide to Methods and Measures. Oxford: Oxford University Press.

Irwin, David E. 2004. Fixation location and fixation duration as indices of cognitive processing. In J.M. Henderson & F. Ferreira (Eds.), The interface of language, vision, and action: Eye movements and the visual world, 105-133. New York, NY: Psychology Press.

ITC Guidance on Standards for Subtitling. Online at: http://www.ofcom.org.uk/static/archive/itc/itc_publications/codes_guidance/standards_for_subtitling/subtitling_1.asp.html

Jensema, Carl. 2000. Eye movement patterns of captioned TV viewers. American Annals of the Deaf vo. 145, no. 3, 275-285.

Karamitroglou, Fotios. 1998. A Proposed Set of Subtitling Standards in Europe. Translation Journal 2(2). http://translationjournal.net/journal/04stndrd.htm

Krejtz, Izabela, Agnieszka Szarkowska, and Krzysztof Krejtz. 2013. “The Effects of Shot Changes on Eye Movements in Subtitling.” Journal of Eye Movement Research 6 (5): 1–12.

Kruger, Jan-Louis and Faans Steyn. 2014. “Subtitles and Eye Tracking: Reading and Performance.” Reading Research Quarterly 49 (1): 105–120.

Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2013a. “Measuring the Impact of Subtitles on Cognitive Load: Eye Tracking and Dynamic Audiovisual Texts.” Proceedings of Eye Tracking South Africa 29-31 August 2013, Cape Town.

Kruger, Jan-Louis, Esté Hefer, and Gordon Matthew. 2013b. The impact of subtitles on academic performance at tertiary level. Paper presented at the Linguistics Society of Southern Africa annual conference in Stellenbosch, June, 2013.

Kruger, Jan-Louis. 2013. “Subtitles in the Classroom: Balancing the Benefits of Dual Coding with the Cost of Increased Cognitive Load.” Journal for Language Teaching 47(1):29–53.

Kruger, Jan-Louis, Hefer, Esté, and Gordon Matthew. 2014. Attention distribution and cognitive load in a subtitled academic lecture: L1 vs. L2. Journal of Eye Movement Research 7(5):4, 1–15.

Langton, Stephen R.H., Anna S. Law, Burton, A. Mike and Stefan R. Schweinberger. 2008. Attention capture by faces. Cognition, 107:330-342.

Pavakanun, Ubowanna. 1992. Incidental acquisition of foreign language through subtitled television programs as a function of similarity with native language and as a function of presentation mode. Unpublished doctoral thesis, Leuven, Belgium, University of Leuven.

Perego, Elisa, Fabio Del Missier, Marco Porta and Mauro Mosconi. 2010. “The Cognitive Effectiveness of Subtitle Processing.” Media Psychology 13(3):243–272.

Rajendran, Dhevi, Andrew Duchowski, Pilar Orero, Juan Martínez, and Pablo Romero-Fresco. 2013. “Effects of Text Chunking on Subtitling: A Quantitative and Qualitative Examination.” Perspectives: Studies in Translatology 21(1):5–31.

Rayner, Keith. 1984. Visual selection in reading, picture perception, and visual search: A tutorial review. In Attention and performance edited by H. Bouma and D. Bouhwhuis, vol. 10. Hillsdale, NJ: Erlbaum.

Rayner, Keith 1998. “Eye movements in reading and information processing: Twenty years of research.” Psychological Bulletin, 124:372–422.

Robson, Gary D. 2004. The closed captioning handbook. Amsterdam: Elsevier.

Romero Fresco, Pablo (in press) The Reception of Subtitles for the Deaf and Hard of Hearing in Europe. Peter Lang.

Smith, Tim, and John M. Henderson. 2008. Edit Blindness: The relationship between attention and global change blindness in dynamic scenes. Journal of Eye Movement Research 2(2), 6:1-17.

Specker, Elizabeth, A. 2008. L1/L2 Eye Movement Reading of Closed Captioning: A Multimodal Analysis of Multimodal Use. Unpublished PhD thesis. University of Arizona.

Szarkowska, Agnieszka, Krejtz, Izabela, and Łukasz Dutka. (forthcoming) The effects of subtitle presentation rate, text editing and type of subtitling on the comprehension and reading patterns of subtitles among deaf, hard of hearing and hearing viewers. To appear in: Across Languages and Cultures 2016, vol. 2.

Szarkowska, Agnieszka, Krejtz, Izabela, Kłyszejko, Zuzanna and Anna Wieczorek. 2011. “Verbatim, standard, or edited? Reading patterns of different captioning styles among deaf, hard of hearing, and hearing viewers”. American Annals of the Deaf 156 (4):363-378.

Vanderplank, Robert. 1988 “The value of teletext sub-titles in language learning”. ELT Journal 42(4):272-81.

Velichkovsky, Boris M., Joos, Markus, Helmert, Jens R., and Sebastian Pannasch. 2005. Two Visual Systems and Their Eye Movements: Evidence from Static and Dynamic Scene Perception. InCogSci 2005: Proceedings of the XXVII Conference of the Cognitive Science Society, 2283–2288.

Winke, Paula, Susan Gass, and Tetyana Syderenko. 2013. “Factors Influencing the Use of Captions by Foreign Language Learners: An Eye Tracking Study.” The Modern Language Journal 97 (1):254–275.

Yarbus, Alfred L. 1967. Eye movements and vision. New York, NY: Plenum Press.

 

Notes

[1] Values are taken from a number of sources and vary depending on a number of factors (see Rayner, 1984)

 

Bios

Jan-Louis Kruger is director of translation and interpreting in the Department of Linguistics at Macquarie University in Sydney, Australia.  He holds a PhD in English on the translation of narrative point of view. His main research interests include studies on the reception and cognitive processing of audiovisual translation products including aspects such as cognitive load, comprehension, attention allocation, and psychological immersion.

Agnieszka Szarkowska, PhD, is Assistant Professor in the Institute of Applied Linguistics at the University of Warsaw, Poland. She is the founder and head of the Audiovisual Translation Lab, a research group working on media accessibility. Her main research interests lies in audiovisual translation, especially subtitling for the deaf and the hard of hearing and audio description.

Izabela Krejtz, PhD, is Assistant Professor at University of Social Sciences and Humanities, Warsaw. She is a co-founder of Eyetracking Research Center at USSH. Her research interests include neurocognitive and educational psychology. Her applied work focuses on pro-positive trainings of attention control, eye tracking studies in perception of audiovisual material and emotions regulation.

Editorial – Seeing Into Things: Eye Tracking the Moving Image – Sean Redmond & Craig Batty

Seeing into Things

We chose Seeing into Things: Eye Tracking the Moving Image as the title of this special edition to foreground the importance of reaching beyond – and beneath – the surface of the screen and the worlds that it creates and envisions. Through the empirical data that eye tracking affords us we are able to evidence and account for the depth in perception and sensibility that accompanies or anchors viewing. Seeing into Things is also recognition of the layers – or epidermi – of technological vision: depth cues, focal length, camera movement, and the delicious qualities of mise-en-scene all invite or demand that the image is looked into. There is much to observe across the textures and texturality of the screen. Eye tracking technology sees into the eyes of the viewer who peers – pierces – into the immersive world of the screen, factual or fictional. We see beauty in this alignment between the eye tracker, the viewer, and the screen. As this special edition finds, Seeing into Things is an enriching and intoxicating way of (re)discovering the complexities of viewing the moving image.

The poetry in and of seeing is not simply experiential, but connected to neurological, anatomical and cognitive processes. It is also connected to culture, discourse and ideology, where seeing into things is always gendered, classed and raced, amongst other encultured practices and modes of being in the world. Seeing into Things enables us to see into ourselves and into the complex and sometimes messy relationships between biology and culture, the human and technology, and between eye, brain, body and ear.

This last carnal conjunction is essential to the work being undertaken in this special edition because Seeing into Things is also meant to critically draw attention to the ocularcentric way through which the world is presently imagined to be experienced. Our position here is not to support this insightful supremacy, but rather to offer challenges and counter-points to it. When we see into things in this special edition it is with the need to recognise the centrality of hearing to seeing; of touching to viewing; and of the incorporation of the full human sensorium as it is taken up and in, and extends itself towards, the screen worlds that move and affect it. As Vivian Sobchack (2000) observes:

As “lived bodies” (to use a phenomenological term that insists on “the” objective body as always also lived subjectively as “my” body, diacritically invested and active in making sense and meaning in and of the world), our vision is always already “fleshed out”–and even at the movies it is “in-formed” and given meaning by our other sensory means of access to the world: our capacity not only to hear, but also to touch, to smell, to taste, and always to proprioceptively feel our dimension and movement in the world. In sum, the film experience is meaningful not to the side of my body, but because of my body.

The eye, brain, body and ear conjunction is also recognition that in order to understand viewing processes, one needs to incorporate different academic disciplines and approaches; from the vision sciences, neuroscience and linguistics; from ethnography and anthropology; and from the arteries and veins of creative practice, to the orbital concerns of the phenomenological. To do our work properly, then, Seeing into Things requires the eyes, brains, bodies and ears of scientists, anatomists, anthropologists, musicologists, filmmakers, screenwriters, and screen theorists, amongst others. It is this exciting arts-science nexus that this special edition draws uniquely from and is built upon, offering a foundational intervention into the way one makes critical and creative sense of viewers’ engagement with the moving image.

But what are the origins of this interdisciplinary approach? From where did the impetus for Seeing into Things come? Let us now return to the origins of the formation of the research group that drives many of the articles in this edition. Let us set the cinematic mood for some groundbreaking eye tracking research.

In the Mood for Eye Tracking Research

After a screening of the film In the Mood for Love (Kar-wai, 2000), Sean Redmond, a film and television scholar, mentioned to neuroscientist and anatomist Jodi Sita, how its rich colour scheme, expressionistic lighting and meandering narrative had fascinated and affected him. He suggested that he was sure his eyes were focusing on these visual elements, as they were being foregrounded, but also that they ‘wandered’ about the screen, choosing to look at motifs, characters and textures at their own volition, and where ‘mood’ took them. Sean contended that his eyes, or the way he viewed film, were both under the command of the film’s narrative and aesthetics, but were also free to discover the opulent fictive world for themselves. He suggested that viewing is an embodied experience.

Jodi responded quite directly: how do you know this? What evidence do you have? She continued that perception and comprehension are cognitive processes, and that what the eyes attend to in any viewing context can be measured objectively and understood through eye tracking, as well as other physiological technologies such as the measurement of pupil dilation. In that moment an arts-science debate was ignited, and an idea for an empirically driven eye tracking the moving image research group was born. They were now in the mood for some landmark empirical research of the moving image.

Jodi and Sean set up the Melbourne-based Eye Tracking and the Moving Image Research group at the end of 2012. They had two central goals in bringing the group together: one, they wanted to utilise eye tracking technology more centrally in the analysis and examination of the moving image; and two, they wanted to draw together scholars and practitioners from the Sciences, and the (Creative) Arts and Humanities so that different modes of enquiry, and theoretical and methodological apparatus, were placed in the same analytical arena (see Jodi’s account of the group’s formation in this edition). It was felt that having a room full of filmmakers, artists, film and cultural theorists, screenwriters, visual ethnographers, vision scientists and neuroscientists would generate new and exciting conversations and deliberations about how viewers engage with the moving image. To employ a games analogy, Jodi and Sean felt it was as if we had we all pinned our tails to different parts of the donkey, but that through opening our eyes together, we would all finally get to see and comprehend its full and glorious anatomy.

Their desire was to build upon existing research that drew disparate disciplines together, extending the type of work being conducted in arts-science research centres such as the Department of Psychology, Neuroscience and Behaviour’s NeuroArts Lab at McMaster University, Hamilton, Ontario. The formation of the group created a strong commitment to inter-disciplinary and cross-institutional relationships, and to what was considered a necessary dialogue between different disciplines united by a shared desire: to investigate vision regimes in relation to the affecting power and beauty of the moving image.

The utilisation of eye tracking technology was thus not born out of a technological determinism, but as a tool to bridge and fuse different approaches and methodologies in order that new findings, new knowledge, and new ways of understanding seeing and sensing images could emerge. This approach drew upon work by scholars who had already ‘crossed the line’, so-to-speak, including the work of Uri Hasson, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin, and David J. Heeger; who had already introduced to the field the idea of neurocinematics, the neuroscience of film, and the ‘inter-subject correlation analysis (ISC) … used to assess similarities in the spatiotemporal responses across viewers’ brains during movie watching’ (2008: 1). But you may ask: what is eye tracking?

But what is Eye Tracking?

Eye tracking enables us to empirically measure what viewers look at when watching screen-based media. The technology allows us to gather data from all platforms, interfaces and portals through which the moving image is distributed and consumed, including the television set, the cinema screen, the computer, and mobile devices such as smartphones and tablets. It also enables us to enter different types of environment to record viewing patterns, including the home and public spaces, such as the mall or the commute to work.  Analysis of viewers’ engagement with the moving image includes assessing where they look; interpreting why and how they look within determined visual fields or Areas of Interest (AOIs); and exploring what they feel or experience when they look. One can employ eye trackers to analyse viewer engagement with elements such as narrative, cinematography, editing, aesthetics, sound design and score, and characterization – elements that feature in many of the articles in this special edition. To do so, however, requires not only recognition and understanding of the languages employed in telling moving image stories, but also engagement with the science of the eye and the physiological and cardio-vascular transformations that take place when screen content is being viewed. To this end, a range of supportive investigative and methodological tools is also often employed, including the measurement of pupil dilation and the monitoring of heart and breathing rates.

Eye trackers work by shining infrared light onto the eye, which is then reflected back and captured by a sensor. The way we view images involves rapid eye movements that alter between points of fixation, in which the visual system gathers information and quickly moves between fixations called saccades. The sensor allows these eye movements (fixations) to be tracked, and specialist software then visualises these movements in the form of heat maps, swarms and gaze plot graphs. Statistical data can be extracted from these visualisations, and an interpretative framework can also be employed. For example, heat maps show effectively the weighting of all the viewing that occurred in a given scene, and gaze plots show the location of the fixations as well as the sequence in which they were made. To draw conclusions from this data, an area of interest analysis can be performed in which the number of times viewers visited specific objects or areas can be computed. By then analysing the amount of processing time spent in these areas, researchers are able to consider things such as the number of return visits made, building a picture of what was concentrated on. As can be seen from the articles in this special edition, analysis of this data draws us into open, and sometimes competing, exchanges about what has been discovered and why.

Double Dialogue

The articles in this edition are engaged in what we would like to define as a double dialogue. Each of the articles stands in their own right as discrete research, and yet they are also engaged in reflective and reflexive commentary. This dialoguing happens both within articles (see, for example, Redmond et al.) and also across articles (see, for example, Dyer and Pink, who draw upon the work of Batty et al.), to explore the possibilities and limitations of eye tracking research. The conversations that emerge enable the arts-science nexus to gather its power, since the different approaches to the text and their findings are foregrounded, drawn into syncretic union, or else are openly contested (see, for example, Brown and Smith’s engagement with each other’s work).

One can read this special edition, then, as the literal embodiment of the grounded work that takes place in a shared, respectful and mutually supportive interdisciplinary working environment. The virtues of the double dialogue approach to a special edition such as this are many, but most importantly one can see the value of the research on its own terms, and see how it has grown out of a dynamic research environment. We are able to witness directly how contributors have worked with and for each other, and how they are able to accommodate and enrich each other’s understandings of the texts under investigation. By seeing into things in this way, powerful research stories emerge.

The Stories of Seeing into Things

We have chosen to present the articles in this edition in a way that tells a research story, where conversations emerge and narrative arcs progress within and across the work presented. We have ordered them in a way that creates a narrative pattern; one can see ideas and themes introduced in one article picked up and developed in another. The story is also one that moves across screen media, from film to television and from features to serials. The special edition opens with a master shot of the field and closes with a tying up of the narrative threads that have been presented throughout the special edition. That is not to say, as previously noted, that each article does not stand as discrete research, but to recognise the beautiful truth of bringing overlapping and communicative research stories together like this.

The stories of Seeing into Things are also about the research environment that has been cultivated through the work of the Eye Tracking and the Moving Image Research group, and in the process of putting this special edition together. New international research relationships have been fashioned; and new friends have been made. We find in the inter-disciplinary stories of this special edition a range of content, styles and approaches in a deliberate attempt to engage readers (other researchers and practitioners) in recognising the power of crossing the research line.

Adrian G. Dyer (a vision scientist) and Sarah Pink (a visual anthropologist and ethnographer) open the edition with a critical, holistic overview of eye tracking research in relation to the screen. In Movement, Attention and Movies: the Possibilities and Limitations of Eye Tracking? Dyer and Pink suggest that film narrative and the conditions of viewing have a significant influence on gaze relations and subjectivity, but that there is yet limited work on the complexities and variables of such connections and alignments. Drawing upon their own research fields in vision science and anthropology and ethnography, Dyer and Pink demonstrate the value and importance of inter-disciplinary scholarship to understanding the poetics and politics of viewing the moving image. To make their observations they draw upon research carried out by Craig Batty, Claire Perkins and Jodi Sita, whose article naturally follows in this edition.

In How We Came To Eye Tracking Animation: A Cross-Disciplinary Approach to Researching the Moving Image, Batty, Perkins and Sita draw upon their pilot study of eye tracking a time-lapse montage sequence from the film Up (a study that also included Dyer). In their article, they outline how the ‘research journey’ of their project took shape, and they discuss how each of them came to the study from their individual disciplines: screenwriting, screen studies and neuroscience. They suggest that their own discipline backgrounds initially influenced and shaped both their research methodology and also the analysis of the research findings. However, they then point towards the layering of these approaches, as a way to fully discover how the montage scene under analysis can be best understood. This inter-disciplinary approach is fully taken up in the next article.

In Sound and Sight: An Exploratory Look at Saving Private Ryan through the Eye Tracking Lens, Jenny Robinson, Jane Stadler and Andrea Rassell place emphasis on the connection between looking and hearing; or, seeing and sounding. By focusing on sonic aesthetics that, arguably, direct viewer attention as much as any other film aesthetic, they use a sound-on, sound-off methodology to test their hypothesis. The resulting discussion will be as useful to film practitioners as it is to screen and eye tracking scholars.

The question of utility, or practice, is taken up in Jan Louis Kruger, Agnieszka Szarkowska and Izabela Krejtz’s, Subtitles on the Moving Image: An Overview of Eye Tracking Studies. They look towards new cognitive research horizons in the field of audiovisual translation (AVT). Seeing limitations and weaknesses in the current eye tracking research being conducted on subtitling, they argue that attention needs to be directed to the actual processing of verbal information. Drawing upon data gathered from numerous eye tracking studies, they contend that it demonstrates the way shot changes, language and subtitles impact upon cognitive processes, and how this has implications for subtitling and captioning.

Drawing on her doctoral research, Tessa Dwyer also explores subtitling but in relation to the BBC television series Sherlock. Dwyer’s fascinating article focuses upon its use of post-production (though scripted) free-floating text.  In From Subtitles to SMS: Eye-Tracking, Texting and Sherlock, Dwyer offers an in-depth analysis of viewer engagement with the show, exploring notions of reading vs. viewing, and attraction vs. distraction. Dwyer also draws upon ideas raised by Sean Redmond, Jodi Sita and Kim Vincs in their article, Our Sherlockian Eyes: the Surveillance of Vision.

In this article, Redmond, Sita and Vincs offer us a unique interior dialogue as they each read the eye tracking data gathered through their own discipline filters while also dialoging with each other’s approaches. Each author sees the hands of direction, misdirection, movement, surveillance and relationality in the scene under analysis, with agreement that vision is never simply cognitive or anatomical but multi-modal and haptic. They employ eye tracking data in ways that recognise the phenomenological embedded in the viewing experience, and which can be ‘extracted’ from what are normally seen or interpreted as qualitative findings.

The final two articles in this special edition then engage in a different type of dialogue or debate. In William Brown’s Politicizing Eye-tracking Studies of Film, he draws upon the (short) history of eye tracking and the moving image research, and specifically the work of Tim J. Smith, to demonstrate its theoretical and applied limitations. While Brown sees great value in eye tracking research he draws our attention to its obviousness in terms of telling us what we may already know. Nonetheless, Brown also outlines where the research may or should go and supplies instructive illustrations to help us chart new courses and terrains.

In what is also a critical commentary on the articles contained in this special edition, Tim J. Smith responds to Brown’s article, pointing to what he sees are misconceptions. In Read, Watch, Listen: A Commentary on Eye Tracking and Moving Images, Smith reflects on his own ground-breaking work as he also summarises and problematizes the articles in this edition. Working from a position as a cognitive psychologist and from within a version of neo-formalist film criticism, Smith’s position on eye tracking is persuasive if caroled.

Combined, the articles in this special edition reflect on the past, present and future of eye tracking and the moving image research and include critiques of the very nature of research itself. The case study material that the articles draw from is predominately from mainstream film and television texts but these are explored through new vectors. Unlike Smith’s work, the authors extend their utilization of eye tracking data to consider the cultural, the ethnographic and anthropological, the ideological and the phenomenological, albeit within the house of film and television aesthetics and genre. We hope that other researchers will draw inspiration and insight from the studies undertaken in this edition.

Future Research Directions

As indicated, the Eye Tracking and the Moving Image Research group features practitioner-academics who are interested in how research can be both carried out and disseminated through creative practice. As two of the articles in this edition signal, there is interest and expertise in sound design and scoring, and in screenwriting. Both of these aspects, which speak to the broader gamut of film and television-making practices, have found natural positions within the research undertaken to date, and also feature in two forthcoming book chapters authored by the group’s members. What is of special interest to us in the near future is how we might use these practices to further develop research methods and research outputs. For example, rather than relying on pre-existing moving image texts, what if we were to make our own? How might we use specific practices – sound, screenwriting – in order to influence the eye tracking experiments that we conduct?

One idea is for the group to make one or more short films in order to test patterns of viewer engagement, where narrative and aesthetics are controlled by the researchers, thus becoming a creative practice research variable. Another idea is to analyse eye tracking data alongside aspects such as the score and the screenplay, in order to make original connections between the source text – intentionality – and its reception. We should also consider how the scientific data provided by the research – heat maps, gaze plots, etc. – might be used as the basis of a creative work in and of itself, such as an artwork or another moving image text. Andrea Rassell, Sean Redmond, Jodi Sita and Darrin Verhagen are currently engaged in public projection and installation projects that use the colours, spirals and vortexes of eye tracking data within thematised artworks.

The make-up of the group has also resulted in some interest in how viewers engage spatially and environmentally with moving image texts, posing questions such as: does the viewing environment alter gaze patterns? How might room set-up and screen size change where and for how long people look at a defined area of interest? In this way, the group might seek to add ethnographic methods to the studies that take place, allowing us to add another set of research variables that could produce interesting and original results. Depending on the context, this type of research would also be of use to the screen industry – distributors, cinema groups, screen manufacturers, interior designers, etc.

We are mindful, nonetheless, where others might take eye tracking research. It is already being used in the commercial moving making industry and one of the worries is that it will become a device to reduce production costs as filmmakers use the data to literally paint the screen by numbers. Film and television are artforms, they beautify the world and they enrich our lives. All members of the eye tracking and moving image research group want to employ eye tracking technology to get to know and understand this beauty, and to fully comprehend what the viewer sees, hears and feels when they watch Fred Astaire dance, Ryan Gosling seduce, or Sherlock deduce and detect.

References

Hasson, Uri, Landesman, Ohad, Knappmeyer, Barbara, Vallines, Ignacio, Rubin, Nava, and Heeger, David J. 2008. Neurocinematics: The neuroscience of film. Projections, 2(1), 1-26

Sobchack, Vivian.2000. What My Fingers Knew: The Cinesthetic Subject, or Vision in the Flesh, Senses of Cinema, Issue 5, available at: http://sensesofcinema.com/2000/conference-special-effects-special-affects/fingers/ (accessed 19th January, 2015).

The Ecstatic Gestalt in Werner Herzog’s Cave of Forgotten Dreams — Kevin Fisher

Abstract: Werner Herzog’s Cave of Forgotten Dreams (2010) has been celebrated as the first non-gratuitous use of 3-D: perfectly suited to revealing the interior of the cave and the naturalistic environment in which it is situated, as opposed to immersing the spectator within computer-generated artificial worlds. I will dispute this reading of the film, describing its use of stereoscopy as instead expressive of an anti-naturalistic ecstatic gestalt by appeal to Ágnes Pethő’s concept of intermediality and the film phenomenology of Vivian Sobchack. Moreover, I will read the figural tropes generated through the film’s use of stereoscopy through George Bataille’s analysis of the emergence of human consciousness, which I argue reciprocates a key thematic of Herzog’s filmmaking.

 

Figure 1: Cave of Forgotten Dreams (Werner Herzog, 2010)

Figure 1: Cave of Forgotten Dreams (Werner Herzog, 2010)

Introduction

This essay began with the desire to read the use of “3D” stereoscopic imagery in Cave of Forgotten Dreams (Werner Herzog, 2010) as an expression of one of the film’s central themes: the enigmatic consciousness of those early humans who left their renderings and cultural artifacts in the Chauvet cave in southern France. Indeed, the film, and Herzog as narrator, both reflect on the cave paintings as a form of proto-cinema, and reciprocally upon cinema as an analog of primitive consciousness. In its reflexive layering of media forms and metaphors between the bookends of what Herzog claims to be the oldest know examples of human representation and the most current cinematic technologies, the film engages in what Ágnes Pethő, in her book Cinema and Intermediality: The Passion for the In-Between (2011), describes as elaborate forms of mirroring characteristic of “abysmal intermediality”. For Pethő, intermedial confrontations open onto an in-between space (or abyme) that transcends medium specificity but instead foregrounds the embodied situations that negotiate them. In this respect, intermedial analysis has phenomenological implications, in relation to which, like Pethő’s own work, I will invoke the film theory of Vivian Sobchack. In describing the structure of the particular abyme onto which Cave opens, I will also draw upon Georges Bataille’s anthropological speculations about the co-emergence of human consciousness, tool use and the order of things to reflect upon the meaning of the particular correlation in which the film configures spectator, object and world, which I will elaborate as its ecstatic gestalt.

Intermediality and Phenomenology

For Pethő, what differentiates intermediality from intertextuality, as well as from more closely related theories of remediation, is the former’s emphasis on embodiment as the primary axis of analysis. As she asserts: “While in intertextuality we have an object that dissolves into its relations, in cinematic intermediality we seem to have moved closer to … a quasi-palpable, corporeal entity in its intermedial density” (2011, 47). In other words, the intermedial resides at the intersections among the embodied situations implicated both by the different media invoked within a given film, and the specific embodied situation of the spectator. She argues further, that as a result of it’s irreducibly embodied nature, “the intermedial cannot be read, at least not in any conventional way that we understand reading … because it is not textual in nature” (67). She continues: “It is not something one ‘deciphers’, it is something one perceives or senses” (68). This assertion involves some contentious assumptions regarding the relationship between embodiment and signification in the context of Pethő’s adaptation of Sobchack’s phenomenology. I’ll return to this issue towards the end of the essay, as it will be exemplified in the analysis that follows. However, I want to assert from the outset that notwithstanding this concern, it is the general phenomenological commitment of Pethő’s theorisation of the intermedial that marks its particular applicability to Cave. Reciprocally, I hope that my reading of the film will also lend some clarification to an approach that is at times as vague as it is provocative. For as Pethő herself acknowledges, “the possible import of phenomenological approaches to film in the interpretation of cinematic intermediality has not been stressed enough…. The phenomenology of intermediality, although hinted at … is yet to be spelled out” (69).

As suggested above, such a “spelling out” must attend to the modalities of embodiment implicated in cinematic intermediality. On one hand, intermediality is itself predicated upon an understanding of film and other moving image based media as intrinsically expressive of situations of embodied consciousness. On the other hand, as Pethő points out: “‘reading’ intermedial relations requires more than anything else, an embodied spectator” (69). In elaborating both sides of this correlation, Pethő relies on Sobchack’s phenomenology of film experience: in the first instance her concept of “film’s body” (1992), and in the second her notion of the spectator as “cinesthetic subject” (2004).

According to Sobchack, “a film must constitute an act of seeing for us to be able to see it” (1992, 129). Insofar as “vision is an act that occurs from somewhere in particular; its requisites are both a body and a world” (25), an observation she applies “not only to the spectator of the film, but also to the film as spectator” (49). What she calls the “film’s body,” like our own bodies, is experienced primarily and prereflectively not as a visibly represented body-object, but as the implicit means of perceiving a visible world. She writes:

Each film projects and makes uniquely visible not only the objective world but the very structure and process of subjective, embodied vision—hitherto only directly available to human beings as the invisible and private structure we each experience as “my own”. (298)

Figure 2: Cave of Forgotten Dreams (Werner Herzog, 2010)

Figure 2: Cave of Forgotten Dreams (Werner Herzog, 2010)

In this respect, the spectator’s own embodied situation functions as the common denominator through which the film’s body is made sensible and intelligible within a “double occupancy of vision” (260). As Sobchack argues, the fact that cinema communicates directly through sound and vision alone does not mean that the film experience is reduced to those channels of perception. Rather, through the phenomena of synaesthesia, the lived body automatically and prereflectively transcodes visual and auditory perceptions indirectly across the other sensory registers. In Cave this is most apparent in the way the proximate relation of the camera to the walls of the cave invokes the tactile qualities of the surface within the lived body of the spectator—as if touching through one’s eyes. The cinesthetic subject is thus neither disembodied—reduced to a transcendental gaze—nor is its experience of film equivalent to direct unmediated perception.

In qualifying this mingled “intermedial density” as “quasi-palbable”, Pethő seems to corroborate Sobchack’s point that the film’s body is never experienced as identical to that of the spectator, nor collapsed or conflated in experience. As she asserts, “I never merely ‘receive’ the film’s vision as my own … ” (271). While they might overlap and intersect in ways uniquely enabled by their intermediation, each still retains a mutual exteriority or otherness so essential to the preservation of that “in-between space” on which the intermedial relies. For Pethő it is precisely the unique openness of the film’s body to that of the spectator combined with the distinct differences between the two that marks the intermedial nature of film experience. She cites Jennifer Barker’s reflections on Sobchack’s phenomenology in this respect:

We exist—emerge really—in the contact between our body and the film’s body… a complex relationship that is marked as often by tension as by alignment…. so that the cinematic experience is the experience of being both “in” our bodies and “in” the liminal space created by that contact. (2009, 19)

What I will explore in this essay is how the use of stereoscopy in Cave of Forgotten Dreams augments this tension, both in relation to the embodied situation of the spectator and the embodied situations implicated by other medialities within the film.

Pethő is equally concerned with intermedial relations within film insofar as the incorporation of other media forms involves exchanges among the modes of embodiment implicated by each. In an article not mentioned by Pethő: “The Scene of the Screen: Envisioning Photographic, Cinematic, and Electronic Presence” (2004), Sobchack prefigures this move as she traces how the “techno-logic” of each medium implicates a particular “phenomo-logic”:

Insofar as the photographic, the cinematic, and the electronic have each been objectively constituted as a new and discrete techno-logic, each has also been subjectively incorporated, enabling a new and discrete perceptual mode of existential and embodied presence. (139)

For example, Sobchack describes a scene in Blade Runner (Ridley Scott, 1982) where Deckard re-animates a photograph belonging to the replicant Leon through a fictional electronic device. She observes how “transmitted to the television screen, the moving images no longer quite retain the concrete, material and objective ‘thingness’ of the photograph, but they also do not achieve the subjective animation of the intentional and prospective vision objectively projected by the cinema” (154). Sobchack’s description of the transmuting force of this remediation anticipates the in-between status that Pethő attributes to the intermedial.

The example from Blade Runner is also indicative of the capacity of film’s body to generate other embodied situations as correlatives of the actual and speculative technologies represented and emulated within its fictional world. In Cave there are a peculiar series of shots, peppered throughout the film, in which scientists researching different aspects of the cave paintings each stand (either alone or with their partners) inside the cave while displaying a printed image of a painting towards the camera. Rather than taking on an enhanced appearance it is the anemic two-dimensionality of the displayed images that is accentuated in relation to the stereoscopically enhanced environment they occupy. While their affect is one of scientific seriousness, the intermedial figuration of these images within the stereoscopic world of the film offers no enhancement of the representational field within their frame, but instead serves to diminish both their objective presence and signifying power. It’s as if stereoscopy is being turned against representation to reverse the relations of containment between the remediated photos and their referents. The insertion of the two dimensional images within the stereoscopically augmented world of the film thereby exemplifies the “abysmal” power of the intermedial to figure one media element against the other as ground, so as to throw some limitation into relief (Pethő 2011, 44).

Pethő describes how intermediality can also generate traces or metaphors from other media that reflexively allegorise embodied situations within the film (65). In one instance, the film’s remediation of the digital image processing techniques of Tossello and Fritz, who in Herzog’s words “used the dimensionality of the surface to create a powerful contrast” by dissecting the palimpsest of marks on the walls (from bear scratches to rendered figures) into discrete layers, offers a reflexive model of his own use of stereoscopy to hold certain intermedial elements in suspension. In another example, the computer-generated model of the cave, a geometric structure of luminous pixels rotating within a blackened virtual space, quite literally turns the unfathomable negative space into a stereoscopic positive—an objective correlative for Sobchack’s description of the film’s body as subjective embodied vision turned inside out and made visible onscreen. However, in Cave the reversibility of film’s body finds itself at an impasse qua abyme relative to the consciousness of the cave painters, which according to Herzog’s voice-over, we “will never be able to understand”.

Figure 3: Cave of Forgotten Dreams (Werner Herzog, 2010)

Figure 3: Cave of Forgotten Dreams (Werner Herzog, 2010)

Stereoscopy and Documentary

One key advantage of Pethő’s account of intermediality as intercorporeality is that it bypasses reductively empirical and technologistic definitions of media specificity while restoring the phenomenological grounds for describing (without reifying) the experiential differences among media. For example, within the context of this study an intermedial approach aids in exploring the tensions between assumptions regarding stereoscopy, documentary, and Hollywood cinema that, according to Barbara Klinger, many of the critical celebrations of Cave of Forgotten Dreams have sought to neutralise. For instance, emphasis has been placed on the fact that “Cave … focused on a real-life marvel rather than a CGI-manufactured landscape” and that it was shot in 3-D as opposed to other films “converted in post-production and thus considered as ‘fake 3D’” (Klinger 2012, 38). On this basis the film is regarded as “one of the few justifiable recent excursions into 3D” (Hoberman 2011), “necessary” for revealing the interior of the cave and the natural environment in which it is situated (Klinger 2012, 38). It’s worth noting how the criteria of indexical documentary realism implicated in the aforementioned defenses of Cave, on the one hand, and the connotations of 3D with Hollywood spectacle, on the other, constitute a problem that must be resolved by these same critics through the recuperation of Cave’s “naturalism”.[1] However, such apologies ring false in the face of Herzog’s own rejection of the documentary tendency (of which he accuses Cinéma Vérité in particular) that “confounds fact and truth” (2002, 301). Instead, he insists that “there are deeper strata of truth in cinema, and there is such a thing as poetic, ecstatic truth. It is mysterious and elusive, and can be reached only through fabrication and imagination and stylization” (301). Thus, “ecstatic truth … is the enemy of the merely factual” and a counterpart rather of the sublime (2010). The notions of media specificity that haunt the critical reception of the film also conspire to diminish what Pethő identifies as the productive tensions of the intermedial both within the film, and between film’s body and spectator. It is thus no surprise that the radical intermediality of Cave must be quelled in order to preserve a received sense of its “documentariness”.

Klinger herself is suspicious of this critical mobilisation of stereoscopic CGI in the Hollywood blockbuster as foil for Cave’s “naturalism”. As she writes:

Cave’s relationship to 3D is more paradoxical and interesting than such contrasts suggest…. In fact [the] film is [reflexively] as much about 3D as it is about its archaeological site…. [Specifically] the stylistic choices of deep focus cinematography (which presents foreground, middle ground, and background in focus) and a dynamically mobile camera help to wed spectacular natural phenomena and the spectacle of space. (39)

Her strategy is thus to argue that Cave transcends the presumed antagonism between the conventions of naturalism and spectacle qua stereoscopy by becoming reflexive in its spectacular naturalism. In so doing, Klinger responds to, but also reproduces, a tendency to understand stereoscopy as mimetically reflecting [or enhancing] structures and properties belonging to the objective world. Contrary to this tendency, and consistent with intermedial analysis I’d like to focus instead on the role of stereoscopy in its production of a mode of subjective embodied consciousness. In this context it is worth recalling Sobchack’s fundamental insight that any cinematic representation of a visible world entails the primarily invisible structuring presence of an embodied viewing subject—what she calls film’s body.

In pursuing the question of what structure of consciousness is manifested by the use of stereoscopy within Cave, one source for an answer can be found in Jonathan Crary’s Techniques of the Observer where he argues that the “lack of planar unity” in the nineteenth century stereoscope produced a fragmented observer quite distinct from the unified observer of the camera/photograph (1990, 128). I want to suggest that the reflexivity of Cave hinges upon a structurally analogous “structural/planar disunity of the perceptual field” (128) that prevails in normative cinematic experience, and a correlative fragmentation of the viewing subject/spectator (though to different effect). Following Crary’s account of the experience of stereoscopic photography, my analysis proceeds from a rejection of the naturalism ascribed to Herzog’s deployment of stereoscopy. Instead, I want to argue that the combination of stereoscopy with deep focus cinematography and camera movement (especially inside the cave), which Klinger refers to as “gold standards … of achieving ‘3Dness’” (2012, 40), often result in a hyperbolic space that exaggerates the separation of figure from ground towards the center of the frame while attenuating it at the peripheries through anamorphosis. The effect is especially amplified during instances of negative parallax (objects appearing to protrude through the screen) in addition to the film’s more persistent positive parallax (the appearance of supplementary depth behind the screen).[2] This emphatic demarcation of figure from ground produces what I describe as the film’s ecstatic gestalt. It is ecstatic in the double sense of ekstasis: as a literal standing out of figure from ground, and as the existential sense of what Herzog describes as “a person’s stepping out of himself into an elevated state” (2010). These two meanings of the ecstatic also relate to the two principle correlations within the gestalt: figure and ground within the perceptual field, and subject figured against the perceptual field as object of perception.

Figure 4: Cave of Forgotten Dreams (Werner Herzog, 2010)

Figure 4: Cave of Forgotten Dreams (Werner Herzog, 2010)

I want to argue that stereoscopy exaggerates the configuration of figure/ground relations within the cinematic image that, according to Sobchack, provides the most fundamental expression of the intentional activity of the film’s body as viewing subject. As she writes: “perception—as an irreducible correlation of figure and ground—forms and organizes a perceptual field(1992, 70). In this respect, every relation of figure and ground implies the intentional activity of a viewing subject as an irreducible element within the gestalt (figured against the perceived world), whose literal and intentional movement reconstitutes the perceptual field, altering and potentially reversing the relation of figure and ground within it. Because this changing configuration of subject/figure/ground is accomplished by “the radical and prereflective deliberation of the body-subject” (Sobchack 1992, 70), we are typically unaware of the co-constituting force of our embodied intentionality, which remains latent to consciousness. However, for Crary this latency is counteracted in stereoscopic photography, which impressed upon consciousness the productive power of the apparatus/observer couplet as projective of dimensionality—experiencing it where it did not objectively exist: within the representational field of the two dimensional image (1990, 129). Or, as Martin Jay summarises: “its three-dimensional images were only in the perception of the viewer—the stereoscope called into question the assumed congruence between the geometry of the world and the natural geometry of the mind’s eye” (1994, 152).

The highly mobile camera in Cave redoubles this effect by associating the exaggeration of its stereoscopic demarcation of figure/ground with the intentional movements of a viewing subject rather than with some natural geometry inherent to the represented world. Put simply, the effect follows the movement of the camera, as in the three-hundred and sixty degree pan around the so-called “cave of the lions” where the sense of added stereoscopic depth, or positive parallax, fluctuates with the distortions of perspective caused by the rotation of deep focus cinematography within such a tight space. The awareness of stereoscopic perception as constituted by the visual subject is also reinforced inside the cave by the spotlight on Herzog’s helmet (or that of the camera person), which creates a sort of iris within the image that circumscribes a zone of greatest effect. Taken together, these representational strategies install a persistent reflective awareness of the productive power of the film’s stereoscopic body within the otherwise prereflectively constituted visual field.

The expressionism of this ecstatic gestalt is also critical to recognition of the intermedial fissures opened in-between the spectator and film’s body. Barker’s earlier remarks to this effect corroborate Sobchack’s argument that the “double occupancy” of the film experience also creates a potential site of tension since the film’s body “in its visible and visual intentional activity, exists within our vision but not as our vision” (1992, 142). As such, “in so far as the visual space I see before me is not completely isomorphic with the bodily space from which I see, there will be a pressure from, an echo of, the machine that mediates my perception” (179; original emphases). This phenomenon of “echo focus” takes on a persistent quality in Cave given that the 3D effect is unlike both non-stereoscopic film and the extra-cinematic space lived by the spectator. The ecstatic gestalt is thus experienced not just as a transformation of the ordinary dimensionality of film, but also of the quotidian three-dimensional world of the spectator.

The “inner landscape” of stereoscopy

Herzog refers to the cave paintings in voice-over as “inner landscapes … of long forgotten dreams”. The notion of an inner landscape is a potent phenomenological and intermedial metaphor which Eric Ames has analysed as exemplary of how Herzog’s representation of “[outer] landscapes serve to conjure unseen words of affect and spirituality, even as they represent the physical world we inhabit” (2009, 58). For example, in his voiceover narration for The Dark Glow of the Mountain (Werner Herzog, 1984), Herzog remarks: “We weren’t so much interested in making a film about mountain climbing per se, or about climbing techniques. What we wanted to find out was what goes on inside mountain climbers who undertake such extreme endeavors…. Aren’t these mountains and peaks like something deep within us all?” During Herzog’s voice-over the camera pans across the peaks and valleys of the range, providing what Ames describes as a “graphic representation” (58) of the affective highs and lows of the climbers’ inner landscape. In Cave’s ecstatic gestalt, I want to locate a related system of correspondence. Though in this case the graphical axis of the inner landscape has been rotated from the vertical and horizontal (peaks and valleys) to the perpendicular (figure and ground), and from the syntagmatic to the paradigmatic.

Figure 5: The Grizzly Man (Werner Herzog, 2005)

Figure 5: The Grizzly Man (Werner Herzog, 2005)

In Cave as in other film’s such as The Grizzly Man (2005), the expression of inner landscapes connects with another persistent theme from Herzog’s oeuvre: exploration of the boundary between human and pre-human or animal consciousness. The relationship of gestalt philosophy to such themes is well established. In his reading of gestalt theorist Jean Piaget, Habermas describes how the emergence of a reflexive capacity within figure/ground relations in the development of the human child (to constitute oneself as figure against the world) recapitulates a stage in the evolution of human consciousness (1979). This is not to deny that infants, pre-human ancestors, or animals possess some ability to delineate figure from ground. The assumption rather is that they are incapable of bringing the gestalt to reflection, and of thereby figuring themselves within it. As Georges Bataille observes in Theory of Religion, although “the animal can be regarded as a subject for which the rest of the world is an object, it is never given the possibility of regarding itself in this way” (1989, 19).[3] Rather, for Bataille, “the animal is in the world as water is in water” (23). For example, the difference between the eater and eaten within the animal world is never qualitative but only ever quantitative: “In the movement of the waters he is only a higher wave overturning the other, weaker ones” (18-19). The animal is in a state of immanence, intimacy and immediacy within a world defined by continuity.

It is only through the correlated emergence of tool use, language and representation that human consciousness becomes reflectively aware of itself within a world of atomized and discontinuous ‘things’ defined by their function within a scheme of utility. For Bataille, the birth of the tool is intricately connected to that of the object and the subject. An instructive rendering of this event can be found in “The Dawn of Man” sequence from 2001: A Space Odyssey (Stanley Kubrick, 1968) when the hominid has an epiphany in which it suddenly perceives/conceives a bone lying on the ground as a weapon. Simultaneous with this realisation is its ability to imaginatively extrapolate the application of the tool into other situational contexts (from breaking other bones on the ground, to smashing the head of an animal, to killing the leader of a rival group at the water hole), as well as into hypothetical temporalities, such as planning future uses of the tool. As a result, discontinuity is introduced into the world by an object that is perceived indirectly according to what it does rather than directly in its immanence and immediacy: “the purpose of a plow is alien to the reality that constitutes it” (Bataille 1989, 41). In what Bataille describes as “one of the most remarkable and fateful aberrations of language […] men situated on the same plane where the things appeared (as if they were comparable to the digging stick or the chipped stone) elements that were nonetheless continuous with the world, such as animals, plants, other men, and finally, the subject determining itself” (28, 31). So in other words, the question of what a thing is automatically devolves to a question of its use value, so that all things refer in their utility to humanity, and humanity in turn to God. Through a certain contagion of thought, “the transcendence of the tool and the creative faculty connected with its use are confusedly attributed … to the entire world” (32).

For Bataille, “the world of things is perceived as a fallen world. It entails the alienation of the one who created it…. The tool changes nature and man at the same time: it subjugates nature to man, who makes and uses it, but it ties man to subjugated nature” (41). The subjugation lies in the fact that, in contrast to the reversibility of figure and ground within the discontinuous world, the figuration of the discontinuous human world against the ground of the continuous pre-human world is irreversible. The thing becomes a reducing filter within consciousness, as perceptions become indiscernible from the concepts projected, which take on a sort of autonomy inseparable from the world in-itself. As a consequence, states Bataille, “nothing is more closed to us than this animal life from which we are descended…. We can never imagine things without consciousness … since we and imagine imply consciousness, our consciousness, adhering indelibly to their presence” (20). Nevertheless, as Bataille asserts: “There is every indication that the first men were closer than we are to the animal world; they distinguished the animal from themselves perhaps, but not without a feeling of doubt mixed with terror and longing” (35). An anthropologist that Herzog interviews concurs that the cave painters did not merely see the animals they painted as things but as spiritual entities in a relationship characterised by greater “fluidity and permeability” than the modern world. By contrast, for Bataille, “The sense of continuity that we must attribute to animals … derived a new significance from the contrast it formed to the world of things … [and] offered man all the fascination of the sacred world, as against the poverty of the profane tool (of the discontinuous object)” (35). In Cave as in other films such as Encounters at the End of the World (2007), Herzog is most interested in the inner landscapes and unconscious drives of the scientists he interviews, such as Julien Monnet who reports that after going into the cave he couldn’t stop dreaming of lions and paintings of lions, and that he was possessed by “a feeling of powerful things and deep things, a feeling of understanding things, that was not a direct way”.

Indeed, Bataille concurs with Herzog that indirection is the only means of approach to such depths. The “abysmal” quality of this reciprocal mirroring between the human and the animal is illustrated traumatically in Herzog’s Grizzly Man (2005) through the folly of Timothy Treadwell’s desire to “enter the secret world of the bears”. Narrating over a close-up shot of the face of the bear that likely killed and ate Treadwell, Herzog senses: “no understanding, no kinship, no mercy, only the overwhelming indifference of nature. To me there is no such thing as the secret world of the bears and to me this blank stare speaks only of a half bored interest in food” into which Treadwell was (at his own peril) anthropomorphically projecting an “inner landscape”. Hence the irony (and the poetry) that the false opening through which Treadwell imposed himself on the world of the bears was reprised by an open mouth, the only way into the animal world being the reduction of the self to pure immanence, since one could not become animal and maintain any vestiges of human subjectivity and sovereignty.

Here we might begin to grasp the function of Cave’s stereoscopic ecstatic gestalt: to return its spectator not just to the physical site of the cave, but to a consciousness still astonished by the novelty and discontinuity of the thing figured in unnatural and quasi-hallucinatory fashion against a continuity that it occluded, but from which to quote William Wordsworth on his recollections of early childhood, it was “still trailing clouds of glory” ([1919] 2008, 536). In this way, my reading of Cave’s ecstatic gestalt through Bataille’s speculative anthropology also resonates with Paul Arthur’s elaboration of Herzog’s “metaphysical realism” in which:

As self-professed intermediary between opposing worlds—modern/pre-modern, prosaic/myth, accessible/recondite—Herzog’s strongest moments revolve around what can’t be shown, what exceeds or beggars representation … [and to] that which testifies to his own inadequacy and, by extension, that of cinema’s meager communicative tools. (2005, 5)

This metaphysical realism is reflected, for instance, in the way that Cave points from objective to non-objective forms of transcendence, between that from which the camera is merely physically blocked and that which is existentially inaccessible. For example, the reverse side of a large stalactite in the “cave of lions” on which is represented the one arguably human figure in the cave, cannot be viewed directly from the walkway to which the camera crew is restricted. “You’ll have to make do with a partial image”, observes the scientist supervising the filming. Undaunted, Herzog’s crew mounts the camera in reverse on a boom and extends it into the space to capture the opposite side of the formation. The effect of negative parallax is pronounced as a result of the proximity of the camera to the surface, which seems to bulge through the screen. While the attempt to get around the backside of the representation yields a full image, it reveals a partial human: resembling a bison from the waist up and what appears to be a female nude (reminiscent of the Venus of Willendorf) from the waist down. The desire to get behind the image leads paradoxically to an image of the desire: the fundamentally conflicted impulse to merge human and animal modalities of embodied consciousness (via “fluidity and permeability”) through the means of representation.

Cave’s ecstatic gestalt involves both the literal sense of ekstasis as a figural standing out, and the metaphorical/metaphysical sense of an extraordinary pronouncement of being that defies representation. Writing in relation to a group of non-stereoscopic films invested in the expression of the spiritual through the material,[4] Sobchack observes how “the camera seeks a parallel ekstasis in the ‘flesh’ of the world: it offers up a profane illumination of objective matter that … opens into an apprehension of something ultimately unfathomable, uncontained and uncontainable—not only in the thing on which we gaze but also in ourselves” (2004, 298). In Cave the intensity of stereoscopic effects wax and wane relative to the film’s intentional objects. It is significant in this respect that the most pronounced instances of negative parallax accompany the exhibition and demonstration of pre-historic artifacts including weapons and tools, whose figuration within the film re-enacts—and commutes to the spectator—the sudden eruption of discontinuity that, according to Bataille, would have issued from their originary invention. The reflexivity of this intermedial reading is corroborated by the fact that in addition to pre-historic tools, the only similarly ecstatic expressions of negative parallax relate to representations of the tools of filmmaking, such as boom microphones and lighting equipment.

Indeed, negative parallax has a special affinity for tools that extend intentionality into space. However, it is telling that these stereoscopic representations of tools (both old and new) transcend the film’s explanatory function of their utility, and “illuminate” (to use Sobchack’s term) something in excess of their functionality. In this way, inadvertently perhaps, the film also invites intermedial reference to the very gratuitous deployments of stereoscopy from which critics have been so determined to distance Cave. For example, anthropologist Wulf Hein’s demonstration of replica prehistoric spears by thrusting and throwing them into negative parallax finds analogs in Hollywood films from Bwana Devil (Arch Oboler, 1952) to The Hobbit: An Unexpected Journey (Peter Jackson, 2012). However, the effects of negative parallax are most profound in relation to a different category of objects worked by tools but gesturing towards that continuous world whose opening—like a trompe l’oeil—they simultaneously block. The first of these objects is the figure of a chimerical lion/man rotated against a darkened backdrop, and offered as a complement to the image of the bison/woman on the stalactite. The second figure is the Venus of Hollifers, about which anthropologist Nicholas Conard makes the unintentional pun: “this one … stands out. It’s the absolute root of figurative depiction as we know it” (emphasis added). The effect of negative parallax is even greater here than in the previous example. Suspended in a glass box, the small sculpture seems to float in front of the screen in a state of ecstatic discontinuity from the surrounding world. The autonomy and disconnection that these objects achieve relative to their environment resonates with the emphasis on their being the “firsts” of their kind. Through the hyperbolic use of negative parallax, the spectator is invited to return to the moment of their radical “newness” in which their startling discontinuity would have been tantamount to a special effect, like the obelisk before the hominids in 2001.

I don’t offer these examples to imply that Herzog’s use of stereoscopy attempts to directly capture the inner landscape of a pre-human consciousness. For as Bataille declares: “There was no landscape in a world where the eyes that opened did not comprehend what they looked at, where indeed, in our terms, the eyes did not see” (1989, 21). Rather, I want to argue that Herzog cultivates this ecstatic gestalt in order to commute to the film spectator the startling novelty and transformative power that would have accompanied the eruption of the capacity for tool use, representation, and reflective consciousness against the ground of a pre-human world—which is why Herzog refers to the cave as the site of “the birth of the human soul.” In this way, Ekstasis thus accedes to ekphrasis, by which according to Pethő cinema incorporates intermedial relations to point beyond its own limits (46).

Conclusion

In concluding, I’d like to return to Pethő’s assertion that intermedial relations, because they occur at the prereflective level of embodiment, are not textual and cannot be read (69). This position is further grounded in her statement that “phenomenology does not see images as representations or signs; it sees them foremost as events and corporeal experiences” (70). This argument might at first glance seem to concur with that of Bataille. However, where he is content to be silent, Pethő wants description without signification. In this sense, her argument is in contradiction to the existential phenomenology of Sobchack in the context of which she advances it. In The Address of the Eye, Sobchack is explicit that she pursues a “semiotic phenomenology,” meaning that the structure of all systems of signification emerge from and recapitulate the structure of embodied prereflective perception (1992, 8). Thus embodiment appears as a theme only by virtue of being brought to reflection, but it can only be brought to reflection because it is already signifying. Hence for Sobchack, embodiment can be read, and she writes of “a textualizing of the sensing body” (69) which is a correlative of the fact that “in its existential function—perception is always semiotic” (70). The consequence of Pethő’s misreading is a potential mystification of the intermedial and romanticisation of the cinematic, in so far as the intermedial must be non-signifying yet embodied and cinematically expressible.

Relative to this discussion, it is intriguing that where Pethő touches briefly on “3D”, she asserts its antagonism to intermedial phenomena by drawing a contrast between “the intrusive ‘tactility’ of 3D images” and “‘haptic’ images” that, by contrast, “preserve a quality of openness towards intermediality” (105 n. 18). She continues that intermediality depends upon an “aesthetic distance”, which is preserved so long as the film is emulating some other mediality, such as painting or photography—a capacity cancelled by the “illusory display of objects in space that act upon our senses (as in the case of 3D imagery)” (105 n. 18). This critique seems overly proscriptive in light of the intermedial elements clearly apparent in a film like Cave. Also, when read in relation to Pethő’s dissociation of signs from corporeality, one might diagnose that it is precisely stereoscopy’s exaggeration of the signifying power of embodied consciousness through its ecstatic gestalt that troubles her.

There is also another way in which what Pethő refers to as stereoscopy’s “illusory display of objects” reflexively turns back upon the illusion of the object. In this respect, I would argue that Cave’s ecstatic gestalt induces a sort of impromptu phenomenological reduction upon the familiar, everyday world of things whose taken-for-granted dimensional extrusion as discrete and autonomous objects it renders strangely artificial, and quasi-hallucinatory. This is nowhere so apparent as in the opening scene, when the novelty of the stereoscopic effects would seem most pronounced to the unaccustomed eyes of the audience. The camera glides down a row within a vineyard that borders the area of the Chauvet Cave. Against the undifferentiated manifold of nature the vines are doubly objectified: both as “raw” nature “cooked” (to use Claude Lévi-Strauss’s [1969] terminology) into the useful form of a vineyard, and as figures of vision unnaturally extruded from their ground through the instrumental movement of our own language-laden consciousness. It is in this way that the novelty of Herzog’s use of stereoscopy doubles for that of the more primordial innovation of which cinema is itself an extension. By breaking the plane of representation and the illusion of depth to which the cinematic spectator is habituated, Cave simulates the world-rupturing force of a much more fundamental discontinuity in the perceptual gestalt and (importantly) provokes reflexive awareness of this event.

 

References

Ames, Eric. 2009. “Herzog, Landscape, and Documentary.” Cinema Journal 48 (2): 49-69.

Arthur, Paul. 2005. “Beyond the Limits: Werner Herzog’s Metaphysical Realism.” Film Comment 41 (4): 42-47.

Bataille, Georges. 1989. Theory of Religion. Translated by Robert Hurley. New York: Zone Books.

Bataille, Georges. 2005. The Cradle of Humanity: Prehistoric Art and Culture, edited by Stuart Kendall. Translated by Michelle Kendall and Stuart Kendall. New York: Zone Books.

Crary, Jonathan. 1990. Techniques of the Observer. Cambridge: MIT Press.

Habermas, Jürgen. 1979. Communication and the Evolution of Society. Translated by Thomas McCarthy. Boston: Beacon Press.

Herzog, Werner. 2002. “The Minnesota Declaration: Truth and fact in documentary cinema.” In Herzog on Herzog, edited by Paul Cronin, 301-302. New York: Faber and Faber.

Herzog, Werner. 2010. “On the Absolute, the Sublime and Ecstatic Truth.” Werner Herzog: The Only Official and Authentic Website of Werner Herzog. Accessed June 13 2013. http://www.wernerherzog.com/117.html.

Hoberman, J. 2011. “Cave Man: Werner Herzog Can’t Get Out of His Own Way in Forgotten Dreams.” The Village Voice, April 27. Accessed January 12, 2014. http://www.villagevoice.com/2011-04-27/film/cave-man-werner-herzog-can-t-get-out-of-his-own-way-in-forgotten-dreams/.

Jay, Martin.1993. Downcast Eyes: The Denigration of Vision in Twentieth-Century French Thought. Berkeley: University of California Press.

Klinger, Barbara. 2012. “Cave of Forgotten Dreams: Meditations on 3D.” Film Quarterly 65 (3): 38-43.

Klinger, Barbara. 2013. “Beyond Cheap Thrills: 3D Cinema Today, The Parallax Debates, and the ‘Pop-Out’.” Public Journal: 3D Cinema and Beyond 47: 186-99.

Lévi-Strauss, Claude. 1969. The Raw and the Cooked: Mythologiques, Volume One. Chicago: University of Chicago Press.

Pethő, Ágnes. 2011. Cinema and Intermediality: The Passion for the In-Between. Newcastle upon Tyne: Cambridge Scholars Publishing.

Sobchack, Vivian. 1992. The Address of the Eye: A Phenomenology of Film Experience. Princeton: Princeton University Press.

Sobchack, Vivian. 2004. Carnal Thoughts: Embodiment and Moving Image Culture, Berkeley: University of California Press.

Wordsworth, William. (1919) 2008. “Ode on Intimations of Immortality from Recollections on Early Childhood.” In The Oxford Book of English Verse: 1250-1900—Volume II, edited by Arthur Quiller-Couch, 536. Alcester: Read Books.

 

Filmography

Herzog, Werner. 2010. Cave of Forgotten Dreams. Creative Differences.

Herzog, Werner. 2007. Encounters at the End of the World. Discovery Films.

Herzog, Werner. 1985. The Dark Glow of the Mountains. Sudfunk Stuttgart.

Herzog, Werner. 2005. The Grizzly Man. Lions Gate Films.

Jackson, Peter. 2012. The Hobbit: An Unexpected Journey. New Line Cinema.

Kubrick, Stanley. 1968. 2001: A Space Odyssey. USA: Metro-Goldwyn-Mayer.

Oboler, Arch. 1952. Bwana Devil. United Artists.

Scott, Ridley. 1982. Blade Runner. Warner Bros.

 

Notes

[1] Such critical tactics are ironic given Herzog’s self-reflexive stance towards the realist strategies he routinely deploys. Evident here, for example, in his penchant for testing the credulity of the audience, such as the apocryphal story about albino alligators mutated by radiation that concludes Cave of Forgotten Dreams.

[2] For a discussion of debates surrounding the value of negative and positive parallax, see Barbara Klinger, “Beyond Cheap Thrills: 3D Cinema Today, The Parallax Debates, and the ‘Pop-Out’” (2013).

[3] Over the span of his life, Georges Bataille produced a collection of short essays and talks on prehistoric cave art (from 1930 to 1957) compiled in The Cradle of Humanity: Prehistoric Art and Culture (2005). However, his Theory of Religion (1989) reproduces many of these ideas in more systematic form, especially with regards to the relationship between tool use, language, and the emergence of human consciousness.

[4] She refers specifically to Diary of a Country Priest (Robert Bresson, 1950), Thérèse (Alain Cavalier, 1986) and Babette’s Feast (Gabriel Axel, 1987).

Bio: Kevin Fisher is a Senior Lecturer in the Department of Media, Film & Communication at the University of Otago. His research interests include phenomenology, special effects and audio-visual analysis, and documentary. His essays have appeared in the anthologies Meta-Morphing (2000), The Lord of the Rings: Studying the Event Film (2007), Cinephilia in the Age of Digital Reproduction (2008) and The Fourth Eye: Mäori Media in Aotearoa/New Zealand (2014) as well as journals such as Science Fiction Film & Television and The New Review of Film and Television and The New Zealand Journal of Media Studies.

 

‘Out wiv the old ay plumma?’ The Uncanny Marginalized Wastelands of Memory and Matter in David Cronenberg’s Spider – Samantha Lindop

Abstract: The shift of economic focus from industrial production to consumption in contemporary Western Society has meant that once booming factories and their surrounding infrastructure are now redundant. Left to decay, the places and spaces of yesteryear are now derelict wastelands that intrude upon the present as a fractured semblance of the past. Hauntingly familiar yet disturbingly unfamiliar they embody the uncanny, evoking a sense of something that ought to remain secret and hidden but that has come to light. Just as unwelcome memories that exist at the core of the uncanny are repressed, confined to the margins of the mind, so too are discarded places and buildings, resonating messages of a failed past, confined to the margins of society. Both memory and matter are delegated to the status of unwanted waste, left to decompose over time. I argue that the film Spider (David Cronenberg, 2002) represents a powerful journey through the marginalized wastelands of memory and matter. Both the memories of the film’s central protagonist Spider (Ralf Fiennes) and the decomposing landscape surrounding him are inextricably bound in the uncanny, both become disjointed from the sequential structure of time, returning as a fragmented ruin of the past, imposing their disturbing presence on the present, causing the fragile web of Spider’s mind to disintegrate like the decaying wasteland around him. 

Introduction

German Intellectual Walter Benjamin famously writes: “the rags, the refuse – these I will not inventory but allow, in the only way possible, to come into their own: by making use of them” (1999, 82). Abandoned, decaying urban wastelands such as those depicted in Spider (David Cronenberg, 2002) may be rendered useless in contemporary consumer society but it does not mean that their history has ceased. The rags and the refuse that are post-industrial wastelands form an unconscious backdrop to contemporary life, they come back from the past, enforcing their presence on the present as a haunting and subversive symbol of something that ought to remain secret and hidden but has come to light. As such, urban wastelands embody Freud’s description of the uncanny in the same way that disturbing memories – buried deep in the mental wasteland of the unconsciousness embody the uncanny when they re-emerge.

I contend that not only are the derelict wastelands of Spider the physical embodiment of the uncanny, but the memories of the film’s central protagonist, Dennis “Spider” Cleg (Ralf Fiennes) also possess an uncanny quality. Memory and matter interact and amalgamate, both becoming disjointed in time; the past, now fragmented, fragile and decaying, returns to haunt the present with disastrous consequences for the future as the frail web of Spider’s mind falls apart like the dilapidated wastelands around him.

From its inception psychoanalysis has been used to explore representations in cultural texts. Whilst Freud examined art and literature from a psychoanalytic perspective, many contemporary theorists apply psychoanalytic theory to film in order to study unresolved cultural anxieties. To begin I shall explore the way deserted, obsolete remnants of a once thriving industrial age embody a powerful sense of uncanny. I will then discuss the psychological origins of the uncanny from a Freudian perspective and how this links in with contemporary Western society and the post-World War II shift from production to consumption as the primary economic activity. Finally I shall look at how marginalized, neglected places and spaces interact with the deserted, shadowy content of repressed memories in the film Spider.

Released from a psychiatric institution after thirty years of incarceration Spider is sent to live in a dingy, decrepit half way house run by an officious woman called Mrs. Wilkins (Lynn Redgrave). His new home is in the same neighborhood that he grew up in and although it is now run down and derelict, all the old places of his childhood still exist. However, the return to the landscape of his youth begins to undermine the fragile stability of his psyche. Disturbing memories that have remained deeply repressed start to emerge, intruding on the present. Time becomes disjointed as he watches the past playing out before him. As memory and contemporary reality intermingle, the film reveals that Spider, an only child, and his mother have a close, loving bond but the relationship between Spider’s parents is strained. His father Bill (Gabriel Byrne) is volatile, intolerant and resentful of his familial responsibilities. Preferring to spend his time at the local pub, Bill begins an affair with Yvonne Wilkinson (Miranda Richardson), a “cheap tart” who frequents the same bar.

One night Spider’s mother (also played by Miranda Richardson) finds Bill and Yvonne having sex in the allotment shed. Bill murders Spider’s mum with a shovel and buries her in the vegetable patch while Yvonne watches on. Yvonne then moves into the family home replacing Spider’s mother, much to his disgust. Spider knows that they murdered his mother and formulates a plan to murder Yvonne. He attaches string from his bedroom down to the kitchen, affixing it to the gas oven. As Yvonne sleeps in a chair next to the stove he pulls the string that turns on the gas. However, as Bill pulls her dead body from the gas filled house she turns back into Spider’s mother whom she really had been all along.  Her transformation into Yvonne was all in Spider’s psychotic mind.

Uncanny Post-Industrial Wastelands

Around the early twentieth century there was a shift from industrial production to consumption as the central economic activity of Western society. This movement away from production meant that many factories and their surrounding infrastructure became redundant and were abandoned. With no function they transformed into decaying, derelict urban wastelands like the setting depicted in Spider. In the opening scenes of the film, Spider makes his way through the desolate urban wasteland of post-industrial East London. The streets are lined with disused, bricked up Victorian terrace houses that would have once been animate with factory employees and their families. A canal, in the past vital for the transportation of goods, now sits obsolete. Devoid of human activity, organic life comprises solely of weeds, poking up through cracks in the pavement. The only stir comes from the methodical heaving and thudding of a monolithic gas works that dominates the landscape like an enormous steel monster from the Jurassic era.

The ruined and dilapidated places and spaces of post-industrial urban wastelands generate a strangely unsettling feeling that can best be described as uncanny. For Freud the uncanny is a feeling that is “undoubtedly related to what is frightening – to what arouses dread and horror” (The Uncanny 1919, 218), but it is also unique and somehow different; a type of feeling that deserves a special conceptual term because it produces a disturbing dreamlike feeling of familiarity in what is evidently unfamiliar. The uncanny is uncanny specifically because of its familiarity being “that class of frightening which leads back to what is known of old and long familiar” (Freud , The Uncanny 1919, 218) the familiar and the unfamiliar always inhabit each other. The German word for the uncanny is “unheimlich”, which translates directly as “unhomely” in English. When Spider returns to the neighborhood that was once his home he finds that although it is the same place he grew up in it is also very different. Now a wasteland of neglected ruins it exists in the present as a haunting residue of the past. Both familiar and unfamiliar, it has become an alienating unhomely place that Spider does not quite fit into. Philosopher and academic Dylan Trigg describes the ruin as something which:

 Intrudes upon the seamless present, disordering the unmarked line of time by invoking a spatial plane of uncanniness […] It retains the shadow of its old self, but simultaneously radically destabilizes the present (Trigg 2006, 131).

For Trigg, the way that the wasteland, in a sense, returns from the past, “enforcing its presence in the present” (2006, 131) corresponds with Freud’s account of the uncanny. As Freud asserts: “Unheimlich is the name for everything that ought to have remained secret and hidden but has come to light” (1919, 227). According to Trigg:

This coming to light materializes in the untimely quality of the ruin. Having fallen from (active) time, the ruin becomes disjointed from time. The untimeliness is evident in how past, present and future conspires to converge in the ruin. Having outlawed its functional existence, the ruin’s persistence in time disproves outright extinction, so compels an unexpected return (Trigg 2006, 131).

Confined to the status of waste, obsolete structures sit suspended in time, displacing its linear continuity. As they gradually crumble and decay they retain a semblance of what they once were but grow increasingly distant from their original form. Just like Spider’s memories, the ruins of the urban wasteland embody the feeling of uncanniness, transcending time they come from the past but appear in the present as a fragmented, ghostly revelation that informs and shapes the future.

The Origins of the Uncanny

Freud argues that the psychological origins of the uncanny stem from repressed infantile anxieties that ought to remain hidden in the deep recesses of the mind but as the result of external triggers make an unexpected return (The Uncanny 1919, 227). Freud concludes that the uncanny is directly attached to the mythical figure of the Sand-Man, and to the idea of being robbed of one’s eyes. Freud refers centrally to the story of ‘The Sand-Man’ by E.T.A Hoffmann where the Sand-Man is described as:

A wicked man who comes when children won’t go to bed and throws handfuls of sand into their eyes so that they jump out of their heads bleeding. Then he puts the eyes into a sack and carries them off to the half-moon to feed his children (Freud, The Uncanny 1919, 227)

Through psychoanalytic experience, Freud has found that incidents about the eyes and fear of going blind are a substitute for the dread of being castrated which arises in the Oedipal phase of development where the male infant’s desire for his mother gives rise to an overwhelming fear that the jealous and cruel father, a bitter rival for the mother’s affections, will castrate the child as punishment for his intense cravings (The Uncanny 1919, 241). Freud argues that the self-blinding of the mythical Oedipus was simply a mitigated form of the punishment of castration and this is why the predominant characteristic of the uncanny is something that is familiar and long-established in the mind. It has become forgotten only through the process of repression, concealed in the catacombs of the unconsciousness which serves as a mental wasteland, reserved for things that we would rather pretend do not exist. According to Freud, in Hoffmann’s tale the Sand-Man always appears as a disturber of love, taking the place of the dreaded father, at whose hands castration is expected (The Uncanny 1919, 230).

In traditional Freudian expression the traumatic threat of castration forces the boy to give up his mother as a love object and instead identify with the castrating father who plays a role in the formation of the superego. The task of the superego is to repress aggressive impulses that stem from the infant’s Oedipal desires to take the place of the father. In order to render hostile impulses towards the father innocuous, the child internalizes its violence, sending it back to where it came from in the outset – one’s own ego. There it is taken over by a portion of the ego, which sets itself up against the rest of the ego as the superego. Retaining the castigatory character of the cruel and jealous father the superego functions as an overwhelmingly cruel agency that torments the ego with guilt (Freud, Civilization and its Discontents 1930, 132-135). Feminist theorist Christina Wieland contends that while the successful working through of the Oedipal stage of development involves the elevation of the father to the superego the deep trauma resulting from the essential rejection of the child’s mother as an object of affection and desire necessitates her banishment into the unconscious (2002, 36).

Just as Freud draws on the ancient Greek drama of Oedipus by Sophocles in order to contextualize this particular stage of psychical development, Wieland refers to the Greek tragedy of Agamemnon form the trilogy The Oresteia to provide a schema for the distressing removal of the revered mother from the consciousness. According to the legend, Agamemnon, commander of the Greek fleet had been in Troy fighting the Trojan War for many years. Upon his eventual return he bought with him a mistress named Cassandra whom he had captured in Troy. Agamemnon’s wife Clytemnestra, angry that he had acquired a mistress and wanting revenge for his earlier sacrifice of their daughter Iphigenia, murders both Agamemnon and Cassandra. Clytemnestra and her lover Aegisthus rule the kingdom until Clytemnestra’s son Orestes, prompted by the God Apollo, avenges his father’s death by murdering his mother and her lover. However, Orestes is immediately hounded by the Furies; ancient maternal Goddesses who seek revenge for matricide. Orestes is punished for his dreadful deed with insanity. Just as Orestes commits a physical act of matricide, the total repression of the mother is tantamount to psychical matricide (Wieland 2002, 36-38).

The young Spider is undoubtedly in the midst of Oedipal conflict. He clearly adores his loving, gentle mother, cherishing the time they spend alone together. His brooding, ill-tempered father functions as a disturber of love, an unwelcome intrusion who appears to have very little tolerance or regard for his young son. When Spider observes his parent’s being physically affectionate towards each other he becomes enraged, an emotion that is intensified when a moment of blissful admiration for his mother as she models some new lingerie is shattered when she asks Spider if he thinks his father will like it. However, just as Orestes is mentally unable to deal with the horror of his matricidal actions so too is young Spider unable to cope with the reality that he has murdered his beloved mother, his punishment, like the ancient Greek character before him is insanity. Wieland identifies that the manic solution to intense distress is to deny reality. She argues that the manic responds to the act of matricide by constructing a false belief that the mother is not their true parent (Wieland 2002, 39-40). Thus, Spider’s psychotic reaction is to deny reality by instead constructing an alternative where it is not his mother that he has murdered but Yvonne.

Another example of the expression of matricide in the creative medium of film is the Australian black comedy Bad Boy Bubby (Rolf de Heer, 1993). Bad Boy Bubby is set in a derelict post-industrial wasteland that not entirely dissimilar to the mise-en-scene of Spider. Mentally retarded Bubby (Nicolas Hope) has lived in complete isolation in a dark, filthy, windowless, cockroach ridden concrete box with his abusive, dominating mother Flo (Claire Benito) for thirty five years. His mother uses him for sex and tricks him into believing that the outside air is poisonous. To back up the lie she wears a gas mask whenever she leaves their squalid home. The unexpected arrival of Bubby’s Pop (Ralph Cotterill) triggers a jealous Oedipal response in Bubby as well as leading him to question if the outside air really is poisonous. He murders both his mother and pop, asphyxiating them with cling wrap before venturing out into a strange, inhospitable world where his experiences with other people are complicated by the fact that he cannot communicate on any level other than mimicry.

Memory and Matter in a Changing World

Social theorists Ken Gelder and Jane Jacobs identify that whilst Freud’s primary concern was certainly with the psyche, his exploration of the uncanny is also about one’s sense of place in the modern, changing environment and it attends to anxieties which are symptomatic of an ongoing process or realignment in post-war society (Gelder and Jacobs 1995, 174). Sociologist John Carroll explains that the shift away from industrialization towards consumption has had a destabilizing effect on traditional societal structures in both the private and public sphere, particularly in relation to patriarchal hegemony. Consumer culture meant that for the first time the wife/mother became psychologically central to the family, taking the lead role in spending decisions (Carroll 1985, 173).

Additionally, active challenges to the time-honored order of female subordination and male domination initiated by the rise of feminism, the entrance of women into the labor market, the collapse of the nuclear family and gradual elevation of women to positions of real power within government and industry over the decades has led to the popular presumption that masculinity is in crisis; that there is a general feeling among men that they are no longer capable of fully controlling the world, and that their power and authority can no longer be taken for granted (Brittan 1989, 183).  As famous psychoanalytical theorist Jacques Lacan states:

Whatever its future this decline [of the paternal imago] constitutes a psychological crisis. It may even be that the emergence of psychoanalysis itself is linked to this very crisis (Lacan 1938, 45).

Not all contemporary thinkers agree that the psychological crisis referred to by Lacan is entirely inclusive, for example, sociologist Tim Edwards argues that whilst there is conceptual and theoretical support for the argument of a crisis of masculinity, the actual effects of this crisis undoubtedly vary according to class, ethnicity, age, geography, and sexuality, constituting what Edwards refers to as a “partial crisis” as opposed to an overall catastrophe (Edwards 2006, 16). Similarly, academic Arthur Brittan argues “it is problematic to assume that all men are in crisis given that not all men have the same interests, nor do they share the same collective identities” (1989, 183). However, whilst the extent of a crisis of masculinity is a debated topic, there is some consensus that the male identity has been destabilized by cultural shifts creating a profound effect on the psyche by triggering deep feelings of anxiety that originate in the Oedipal phase of development.

These anxieties reveal themselves in the form of the uncanny in the sense that the derelict wastelands of former factory sites serve as a catalyst, as remnants of the past when gender roles were clearly defined. Similarly, the exclusively masculine domain of the factory serves as an emblem for patriarchal hegemony. Just as unwelcome memories are repressed and confined to the margins of the mind, so too are discarded places that resonate with troubling messages of failure, confined to the margins of society. Both memory and matter are delegated the status of unwanted waste, left to gradually decompose over time.

Memory, Matter and the Wastelands of Spider

The marginalized wastelands of memory and matter form a powerful alliance in Spider, embodying the uncanny. Both become disjointed from time, disrupting its continuity, both generate a lurking feeling of unease as something that ought to remain secret and hidden reveals itself. The places from Spider’s past; the streets, the canal, the allotment, Spider’s childhood home, the local pub, and the gas works, all conspire to exhume Spider’s interred memories. The vegetable garden at the allotment is where his mother was buried after being brutally murdered, under the bridge alongside the canal Spider’s father received sexual favors from Yvonne, the local pub is where Bill preferred to spend his free time and also where he met Yvonne, Spider’s old home is where his mother lovingly indulged his obsession with spiders (as well as being the place where she met her real death). All these places still exist, but like the ruin they are shadows of past that threaten to subvert the present.

As Spider explores the landscape of his youth the past, present and future conspire to converge in his mind just as they do in the ruin. His childhood memories resurface from history, imposing their uncanny presence on the present. Spider becomes lost in time as his surroundings return to their original manifestation and the events of the past play out before his eyes. But as Trigg describes, when we encounter objects from our background a change takes place when we recall their origin:

They return to their original spontaneity, and yet are wholly decaying, rotting, and fragile to the touch. The return of the “thing” thus instills a warped time scale. What remains in the ruin is the trace of a past, fragmented and unable to be situated in an overarching narrative, fusing with the ruin’s decay in the present (Trigg 2006, 31).

As the narrative of Spider’s childhood comes to life cracks begin to appear, revealing the fragility of its construction and the fragility of Spider’s mind. Things are not quite right; Yvonne looks disturbingly like Spider’s mother, although she behaves very differently. No one else seems to notice that Spider’s mother is missing and when Spider accuses Bill and Yvonne of murdering his mother Bill, appearing shocked and concerned, asks him if he is daft.

Just as the fragmented traces of the past fuse with the ruin’s decay in the present, the repressed memories from Spider’s past emerge and combine with the present; as the prologue for the movie articulates: “The only thing worse than losing your mind is finding it again” (http://www.spiderthemovie.com). Spider is no longer able to contain the fact that it was he who murdered his beloved mother; that she was actually Yvonne all along. As Trigg asserts: “In spite of the temporal closure of the past, the same past reconfigures and reappears, circumventing the attempt to rationalize it into submission” (2006, 31).

Spider’s psychotic attempt to rationalize his act of matricide “into submission” by denying that it is his mother that he has murdered, instead displacing culpability for his mother’s death onto his father is thwarted by the return of the truth, which, in response to the external triggers of the post-industrial wasteland around him, has reconfigured and reappeared. Like the string that connects the oven door to the murder’s hand, the memory connects the past to the present. Spider’s recollections reactivate his psychosis, which returns with all the vigor and intensity of its original manifestation.  Mrs. Wilkins morphs into Yvonne Wilkinson, looking and behaving exactly as she did when Spider was a young child. The past and the present become trapped in the tangled web of Spider’s mind, just like living become ensnared in the sticky tendrils of the web of his namesake taking their place alongside decaying corpses that linger as a ghoulish testimony of past consumption. In an uncanny act of repetition he sets about murdering Yvonne again (only this time he fails to complete the deed) and is taken away to the asylum, just as he was thirty years ago, as much a fragmented ruin from the past, conflated with the present as the decaying wasteland around him.

Conclusion

No longer of material value, waste is the stuff we discard, the rags and the refuse that is confined to the outskirts of society, forming an unconscious backdrop that despite its marginalization defies outright extinction. Wasted places and spaces, remnant from another era become disjointed from time, transcending its sequential structure. They retain a semblance of their past life but, now distant from their original form, take on a disturbing, haunting quality that embodies the uncanny; a dream like feeling of familiarity in what is evidently unfamiliar. In David Cronenberg’s Spider the wasted debris of the landscape forms a powerful coalition with disturbing memories that have been impounded in the mental wasteland of Spider’s unconsciousness, each providing the other with substance and epistemological value. Like a hurriedly covered corpse in a shallow grave, memory and matter do not remain buried for long, instead they reemerge, tainting the present with their fetid presence, a presence that in turn contaminates the future. Thus, whether it is in the form of disturbing and traumatic memories or architectural constructions that resonate with outmoded ideologies waste becomes a pseudonym for the things we would prefer to forget; the objects and events that are stifled in the shadow recesses of the unconscious. However, as Walter Benjamin succinctly asserts “by their waste shall you know them” (1999, 82), hence, the unconscious is also a place where narratives of truth, unmediated by the ego reside. As a form of cultural expression Spider provides a compelling commentary on the powerful interaction between matter and memory and the hidden anxieties that embody them both.

References

Bad Boy Bubby. 1993. Directed by Rolf de Heer. Australia. South Australian Film Corporation.

Benjamin, Walter. 1999. The Arcades Project. Translated by Howard Eiland and Kevin McLaughlin. Edited by Rolf Tiedemann. Cambridge Massachusetts: Belknap Press.

Brittan, Arthur. 1989. Masculinity and Power. Oxford: Basil Blackwell.

Carroll, John. 1985. Guilt: The Grey Eminence Behind Character, History, and Culture. London: Routledge.

Edwards, Tim. 2006. Cultures and Masculinity. London: Routledge.

Freud, Sigmund. 1919. The Uncanny. In The Standard Edition of the Complete Psychological Works of Sigmund Freud Vol. XVII: An Infantile Neurosis and Other Works. London: Vintage, 2001, 217-256.

__________. 1930. Civilization and its Discontents. In The Standard Edition of the Complete Psychological Works of Sigmund Freud Vol. XXI: The Future of an Illusion, Civilization and its Discontents and Other Works. London: Vintage, 2001, 59-148.

Gelder, Ken and Jane Jacobs. 1995. Uncanny Australia. Cultural Geographies 2: 171-183.

Lacan, Jacques. 1938. Family Complexes and the Individual. Translated by Cormac Gallagher.  London: Anthony Rowe.

Spider. 2002. Directed by David Cronenberg. Canada/England. Capitol Films.

Spider the Movie Official Web Site. 2003. http://spiderthemovie.com (Accessed October 3rd 2010).

Trigg, Dylan. 2006. The Aesthetics of Decay: Nothingness, Nostalgia and the Absence of Reason. New York: Peter Lang.

Vider, Anthony. 1994. The Architectural Uncanny: Essays in the Modern Unhomely. London and New York: MIT Press.

Wieland, Christina. 2002. The Undead Mother: Psychoanalytic Explorations of Masculinity, Femininity, and Matricide. London: Rebus Press.

 

Bio

Samantha Lindop holds a Masters degree in Psychoanalytic Studies from Deakin University and is currently a PhD candidate at The University of Queensland, School of English, Media Studies, and Art History. She is researching the fatale figure in postmillennial neo-noir North American cinema.  Email: slindop1@bigpond.com