5. Another brick in the wall critically assessing vocal markers of neuropsychiatric conditions: bsky.app/profile/alpa... As in many previous efforts, we show that machine learning markers as identified in current practices don't generalize 6/
Come work with us! Our department is hiring an associate/assistant prof in language and speech technology www.ru.nl/en/working-a...
#interspeech #speech #SpeechTech #SpeechScience
Are there cross-linguistic vocal characteristics of schizophrenia? We test current machine learning approaches and show that they do generalize across languages *not even when being trained cross-linguistically*. Excellent thread by @alpar.bsky.social w some ways forward.
New pre-print with @fusaroli.bsky.social on voice markers of schizophrenia out:
www.medrxiv.org/content/10.1...
Thread below 📖
TL;DR: Cross-linguistic generalizability of vocal markers of SCZ is challenging, we need more collaborative efforts and large multi-center and cross-linguistic projects
8/8 🛠️ How can we improve generalization?
•Larger, open datasets capturing linguistic, clinical, and demogr. variability in SCZ to test generalization and modern ML architectures, e.g., LLMs, multimodal models.
•Focusing on fine-grained clinically relevant features to enhance clinical applicability.
7/8 🌍 Why does generalization fail?
• Linguistic differences affect how SCZ symptoms relate to acoustic features
• Clinical heterogeneity limits robustness of ML models trained on small, homogenous samples
• Models biased toward general features, not capturing diagnosis- or symptom-specific markers
6/8 📢 Key Finding #3:
We tested two alternative approach:
1️) Mixture of Experts models (combining predictions from models trained on different languages, Plot 3).
2) Multi-language training set (combin. training data from multiple languages, Plot 4).
❌ Results: Still near chance level (F1 ~ 0.50).
5/8 🚨 Key Finding
✔️#1: ML models perform when trained/tested on the same language (F1 ~ 0.75) (Plot1)
❌#2: But when trained/tested on different languages (e.g., Danish → Chinese), performance drops significantly (F1 ~ 0.50) (Plot 2).
Cross-linguistic generalizability remains a key challenge!
4/8💡What’s the goal?
In this study we build a large cross-linguistic speech corpus (Danish, German, Chinese) of patients with schizophrenia and controls to systematically test whether voice-based ML models predicting schizophrenia generalize across different languages, samples and context: 🧵
3/8 In prior meta-analysis and experim. work (below), we showed that speech marker generalizability might be challenging. The assumption that SCZ speech markers manifest uniformly across heterogeneous samples and contexts must be systematically tested: doi.org/10.1093/schb... doi.org/10.1016/j.sc...
2/8💡Key question ❓
But how well do voice-based machine-learning models generalize across languages and cultural contexts? How well do they generalize across samples with heterogenous clinical features? Are they robust enough to biases for clinical applicability?
1/8 Schizophrenia and machine-learning-based speech markers
🎙️ Schizophrenia is associated with atypical voice patterns, making voice a promising candidate biomarker. Voice-based ML models can indeed predict diagnosis, symptoms and track socio-cognitive and motor features of SCZ with high accuracy.
New pre-print with @fusaroli.bsky.social on voice markers of schizophrenia out:
www.medrxiv.org/content/10.1...
Thread below 📖
TL;DR: Cross-linguistic generalizability of vocal markers of SCZ is challenging, we need more collaborative efforts and large multi-center and cross-linguistic projects
For more work in this line of research:
- do markers of schizophrenia and its symptoms generalize across languages? (voice: doi.org/10.1093/schb... text: doi.org/10.1016/j.sc...; led by
A. Parola) 1/
