bbrel
burton bradstock research labs
current research and development
automatic film content analysis
current research and development
film is a very good example of multimedia: it combines visual, verbal and aural elements in order to tell stories. the complex blend of multimedia and narrative elements presents many challenges for analysis, but we are able to take advantage of constraints that guide the choices made by film makers. all this makes film a fascinating domain in which to explore multimodality and narrative.
there is also a pressing need for automated film content analysis technologies in order to support future generations of film retrieval and browsing technologies.
for film fans, we envisage applications that allow retrieval by 'story similarity', and hypervideo browsing of films based on a link structure that reflects underlying narrative structures.
for film scholars and film production companies, we envisage a new mode of computer-based film analysis based on the visualisation of film structures.
a third application scenario is the semi-automated production of audio description - spoken descriptions of on-screen action provided for visually-impaired film audiences.
my current work in this area is focusing on: (i) the extraction and integration of information from film scripts, audio description, plot summaries and subtitles; (ii) the fusion of information from these text sources with audiovisual features extracted from film data,in collaboration with Centre for Digital Video Processing, Dublin City University.
selected publications
fusing information from video and text
Salway, Lehane, O'Connor (2007), 'Associating Characters with Events in Films', CIVR 2007.
extracting information about film content from texts...
Tomadaki and Salway (2006), ‘Cross-document Coreference for Cross-media Film Indexing’, LREC 2006 Workshop on Crossing Media for Improved Information Access.
Salway,
Vassiliou and Ahmad (2005), ‘What
Happens in Films?’,
IEEE Conference on Multimedia and Expo, ICME 2005.
Salway and Graham (2003), ‘Extracting Information about Emotions in Films ’, Procs. 11th ACM Conference on Multimedia 2003, 4th-6th Nov. 2003, pp. 299-302.
Vassiliou (2006), 'Film Content Analysis: a text-based approach', PhD dissertation, Dept. of Computing, University of Surrey.
Tomadaki (2006), 'Cross-Document Coreference between Different Types of Collateral Texts for Films' , PhD dissertation, Dept. of Computing, University of Surrey.
turning films into hypervideo based on narrative structures...
Salway and Xu (2005), ‘Navigating Stories in Films ’, Dept. of Computing Technical Report CS-05-04, University of Surrey.
Xu (2005), 'Representation of Story Structures for Browsing Digital Video', PhD dissertation, Dept. of Computing, University of Surrey.
audiovisual translation & the production of audio description...
Salway (in press), ‘A Corpus-based Analysis of the Language of Audio Description’, to be published in Selected Proceedings of Media for All, Rodopi.
Lakritz and Salway (2006), ‘The Semi-Automatic Generation of Audio Description from Screenplays’, Dept. of Computing Technical Report CS-06-05, University of Surrey.