Attention estimation and annotation are tasks aimed at revealing which parts of some content are likely to draw the users’ interest. Previous approaches have tackled these incredibly challenging tasks using a variety of behavioral signals, from dwell-time to clickthrough data, and computational models of visual correspondence to these behavioral signals. Today, these signals are leveraged by a myriad of online services to personalize social media, search engine results, recommender systems, and even in supporting critical decision making, such as financial or medical data. However, the signals that all these services are based on are rough estimations of the real underlying attention and affective preferences of the users. Indeed, users may attend to some content simply because it is salient, but not because it is really interesting, or simply because it is outrageous. In contrast, project BANANA will use brain-computer interfaces (BCIs) to infer users’ preferences and their attentional correlates towards visual content, as measured directly from the human brain. We aim for a scientific breakthrough by proposing the first-of-its-kind affective visual attention annotation via brainsourcing, i.e. crowdsourced BCI signal acquisition. First, our approach will allow accurate estimation of user preferences, attention allocation, and --critically-- the affective component of attention, directly measured from the natural and implicit brain potentials evoked in response to users experiencing digital contents. Then, we will utilize the resulting data in a crowdsourcing setting to reveal how multiple users react to different stimuli and how their attention and affective responses are distributed. These collective responses will produce unified, consistent measures as a result. Our technology will be used in several downstream tasks such as segmentation of users' attention while looking at images, identification of key events, and video summarisation. We will pilot BANANA with different user groups to test and prove its effectiveness, using objective benchmarks and evaluation strategies.

Project partners

• University of Luxembourg - Luxembourg (Coordinator)
• Universitat Jaume I - Spain
• University of Helsinki - Finland
• Politechnika Opolska - Poland