In the following we briefly present two current experiments:
(1) We conducted an annotation experiment on health data from a population-based study in cooperation with the University Medicine Greifswald. The triplet annotation task was to decide whether an individual was more similar to a healthy one or to one with a given disorder. We used hepatic steatosis as example disorder, and described the individuals with 10 pre-selected characteristics related to this disorder. We recorded task duration, electro-dermal activity as stress indicator, and uncertainty as stated by the experiment participants (n = 29 non-experts and three experts) for 30 triplets. We built an Artificial Similarity-Based Annotator (ASBA) and compared its correctness and uncertainty to that of the experiment participants.
More information can be found under publications.
(2) In this study and as a follow-up of (1), we investigated how attentiveness of the participants is associated with the difficulty of the annotation tasks they are called to accomplish. We modeled "attentiveness” by using an eye tracker and measuring the pupil size during eye movement. We split the experiment participants in two groups: One group annotated first the obviously easy tasks and later the obviously difficult tasks. The other group annotated first the obviously difficult task and later the obviously easy tasks. We compared the results with the results of 4 domain experts.
Technologies plays an important role for the annotation quality of labeling tasks. Therefore we work at the moment on a Systematic Review on Virtual Reality for Medical Annotation Tasks. Our work investigates benefits offered by VR towards better experience and towards better understanding of annotators.
Kooperationen im Projekt
Tel.:+49 391 6758967