filename     : Wam20c.pdf
entry        : inproceedings
conference   : The 13th International Conference on Educational Data Mining (EDM 2020), Ifrane, MA, 10-13 July, 2020
pages        : 245-256
year         : 2020
month        : 07
title        : Image Reconstruction of Tablet Front Camera Recordings in Educational Settings
subtitle     :
author       : Rafael Wampfler, Andreas Emch, Barbara Solenthaler, Markus Gross
booktitle    : Proceedings of The 13th International Conference on Educational Data Mining (EDM 2020)
ISSN/ISBN    : 978-1-7336736-1-7
editor       : Anna N. Rafferty, Jacob Whitehill, Violetta Cavalli-Sforza, Cristobal Romero
publisher    : International Educational Data Mining Society {(IEDMS)}
publ.place   :
volume       : 
issue        :
language     : English
keywords     : Front Camera Setup, Inpainting, Affective Computing, Classification, Deep Learning
abstract     : Front camera data from tablets used in educational settings offer valuable clues to student behavior, attention, and affective state. Due to the camera’s angle of view, the face of the student is partially occluded and skewed. This hinders the ability of experts to adequately capture the learning process and student states. In this paper, we present a pipeline and techniques for image reconstruction of front camera recordings. Our setting consists of a cheap and unobtrusive mirror construction to improve the visibility of the face. We then process the image and use neural inpainting to reconstruct missing data in the recordings. We demonstrate the applicability of our setting and processing pipeline on affective state prediction based on front camera recordings (i.e., action units, eye gaze, eye blinks, and movement) during math-solving tasks (active) and emotional stimuli from pictures (passive) shown on a tablet. We show that our setup provides comparable performance for affective state prediction to recordings taken with an external and more obtrusive GoPro camera.