Neural representation of speech features during natural dialogue

Abstract

Studying speech in natural environments presents significant challenges for traditional electroencephalography (EEG) analysis approaches. In recent years, encoding models and machine learning techniques have made substantial progress, enabling a shift toward experimental designs that incorporate naturalistic, dynamic stimuli such as speech. This study aims to understand how different speech attributes are encoded in the brain during unscripted natural dialogue. We begin by analyzing low-level features (envelope, fundamental frequency, spectrogram) and then incorporate higher-level features, such as phonemes and phonological attributes. The results show that including high-level features improves the prediction of neural responses from speech across all frequency bands. Moreover, predictions based on phonemes and phonological features suggest that neural sensitivity is consistent with a hierarchical language processing system.