Neurocognitive Disorder (NCD) Detection, 2021-Now Used pre-trained models as text encoders and fed text features into an ensemble of back-end classifiers to perform automatic AD detection. Built a system obtaining the best published AD detection accuracies with automatic speech transcripts on the ADReSS dataset Working on transferring the methods accross language (applying the methods on Cantonese NCD detection data) and involving data augmentation Multimodal Emotion Recognitions, Oct 2020 - May 2021 Used the visual, acoustic and language (text) information to do emotion recognition for speaking videos
Transferred a Res-TDNN model from previous work done on one dataset to another to investigate the impact of the visual feature
Investigated the effectiveness of different fusion methods to combine different modalities of features
Audio-visual Speech Recognition, Jul - Sep 2018 Conducted image pre-processing for a disordered speech recognition task with audio-visual features.
Recognized and extracted mouth region images from videos by OpenCV and dlib
Built an autoencoder to compress visual data