« Projekte
Sie verwenden einen sehr veralteten Browser und können Funktionen dieser Seite nur sehr eingeschränkt nutzen. Bitte aktualisieren Sie Ihren Browser. http://www.browser-update.org/de/update.html
Annotator-Machine-Interaction
Projektbearbeiter:
Anne Rother
Finanzierung:
Haushalt;
This internal project involves experiments that investigate annotator behaviour for difficult tasks. Since 2019, the OVGU team designs experiments in the KMD Experiment Lab to study annotator confidence and its association to annotation quality for labeling tasks. This expertise will be used in the ITN to assist in the process of identifying outliers in the data.

In the following we briefly present two current experiments:
(1) We conducted an annotation experiment on health data from a population-based study in cooperation with the University Medicine Greifswald. The triplet annotation task was to decide whether an individual was more similar to a healthy one or to one with a given disorder. We used hepatic steatosis as example disorder, and described the individuals with 10 pre-selected characteristics related to this disorder. We recorded task duration, electro-dermal activity as stress indicator, and uncertainty as stated by the experiment participants (n = 29 non-experts and three experts) for 30 triplets. We built an Artificial Similarity-Based Annotator (ASBA) and compared its correctness and uncertainty to that of the experiment participants.
More information can be found under publications.

(2) In this study and as a follow-up of (1), we investigated how attentiveness of the participants is associated with the difficulty of the annotation tasks they are called to accomplish. We modeled "attentiveness” by using an eye tracker and measuring the pupil size during eye movement. We split the experiment participants in two groups: One group annotated first the obviously easy tasks and later the obviously difficult tasks. The other group annotated first the obviously difficult task and later the obviously easy tasks. We compared the results with the results of 4 domain experts.

Technologies plays an important role for the annotation quality of labeling tasks. Therefore we work at the moment on a Systematic Review on Virtual Reality for Medical Annotation Tasks. Our work investigates benefits offered by VR towards better experience and towards better understanding of annotators.

Schlagworte

Data Mining, Data Stream Mining, Informatik, Opinion Mining

Kooperationen im Projekt

Publikationen

2021
Die Daten werden geladen ...
2015
Die Daten werden geladen ...
Kontakt

weitere Projekte

Die Daten werden geladen ...