Prof. Dr. Lora Aroyo
VU University Amsterdam

When: Friday 24 October 2014, 16:00

Where: Room D2.2.094 (WU Vienna, Building D2)

Abstract

Human annotation is a critical part of big data semantics, but it is based on an antiquated ideal of a single correct truth. This presentation exposes seven myths about human annotation and dispels the myth of a single truth with examples from our research. A new theory of truth, CrowdTruth, is based on subjective human interpretation on the same objects (in our examples, sentences) across a crowd, providing a representation of subjectivity and a range of reasonable interpretations. CrowdTruth has allowed us to identify the myths of human annotation and paint a more accurate picture of human performance of semantic interpretation for machines to attain. Implementation of the CrowdTruth framework and some further reading can be found at: http://crowdtruth.org

Biography

Lora Aroyo is an associate professor at the Web & Media Group, Department of Computer Science, VU University Amsterdam, The Netherlands. Her research work is focused on semantic technologies for modeling user and context for recommendation systems and personalized access of online multimedia collections, e.g. cultural heritage collections, multimedia archives and interactive TV. She was a scientific coordinator of the NoTube project, dealing with the integration of Web and TV data with the help of semantics, and a number of nationally funded projects, such as CHIP and Agora, dealing with modelling events and event narratives. Currently involved in SealincMedia: Socially-Enriched Accessed to Linked Cultural Media (http://sealincmedia.wordpress.com/) and ViSTA-TV: Linked Open Data, Statistics and Recommendations for Live TV (http://vista-tv.eu/) projects. She has been co-chair of numerous workshops on crowdsourcing, social web and cultural heritage. Lora is actively involved in the Semantic Web community as a program chair for the European and the International Semantic Web Conferences in 2009 and 2011, as conference chair for the ESWC 2010 Conference, and the editorial board of the Semantic Web Journal. She is also actively involved in the Personalization and User modeling community as vice-president of the User Modeling Inc., and the editorial board of the Journal of Human-Computer Studies and the User Modeling and User-Adapted Interaction Journal. In 2012 and 2013 she won IBM Faculty Awards for her work on CrowdTruth: Crowdsourcing for ground truth data collection for adapting IBM Watson system to medical domain (http://crowdtruth.org/).

For more info: 

web: http://lora-aroyo.org,
twitter: @laroyo
slideshare: http://www.slideshare.net/laroyo