EEG-CLIP
Open Access DepositedFinetune CLIP Model for EEG Classification
This project explores how machine learning can help improve EEG diagnosis by building a system that predicts multiple conditions from EEG images. Right now, diagnosing EEGs often takes a lot of time from trained neurologists, which can be costly and slow for patients. A faster, ML-driven approach could make a real difference in accessibility and efficiency. While models like CNNs and DeepNet have been used before, they often depend on expert-crafted features or aren’t openly available. Newer tools like EEG-GPT and CLIP-based zero-shot models show promise, but they don’t support multi-label predictions. This work fills that gap by adapting and fine-tuning CLIP for multi-label EEG diagnosis, with the goal of making advanced, scalable brain health diagnostics more practical and socially impactful. Out of all the methods tested, the CLIP model fine-tuned with LoRA and a mix of contrastive loss and Kullback–Leibler Divergence (KL Divergence) stood out as the top performer. It consistently delivered the best results, outperforming models like EEG-GPT (a zero-shot GPT-based system), CNN ensembles, and graph convolutional networks. The model achieved excellent metrics across the board, including an AUC-ROC of 0.996, AUC-PRC of 0.982, precision and recall around 0.95. For certain labels, such as GPD, the accuracy was even higher. Beyond just accuracy, the model also showed strong robustness and remained well-calibrated, even when tested on altered or degraded EEG spectrograms. These results highlight the potential of a fine-tuned CLIP model to offer both high accuracy and reliability in real-world EEG diagnosis tasks.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.