Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020

Papers, code, and scores for the 2020 Challenge are available on this page.

Citations

To refer to the 2020 Challenge, please cite:

Perez Alday EA, Gu A, J Shah A, Robichaux C, Ian Wong AK, Liu C, Liu F, Bahrami Rad A, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD^* Reyna MA^*. Classification of 12-lead ECGs: The PhysioNet/Computing in Cardiology Challenge 2020. Physiol. Meas. 2021 Jan 1;41(12):124003. doi: 10.1088/1361-6579/abc960 to refer to the 2020 Challenge.

Please also cite the standard PhysioNet citation. You can find followup articles to the 2020 Challenge in the 2021 Challenge and in the Journal of Physiological Measurement Focus Issue on Classification of Multilead ECGs.

Challenge Results

The conference papers for Computing in Cardiology 2020 are available on the CinC and IEEE websites.

The ranks of the teams and their papers and entry code are available in the table below.

Rank	Team name	Team member(s)	Paper	Code
1	prna	Annamalai Natarajan, Yale Chang, Sara Mariani, Asif Rahman, Gregory Boverman, Shruti Vij and Jonathan Rubin	A Wide and Deep Transformer Neural Network for 12-Lead ECG Classification	Link (86MB)
2	Between a ROC and a heart place	Zhibin Zhao, Hui Fang, Samuel Relton, Ruqiang Yan, Yuhong Liu, Zhijing Li, Jing Qin and David Wong	Adaptive lead weighted ResNet trained with different duration signals for classifying 12-lead ECGs	Link (30MB)
3	HeartBeats	Zhaowei Zhu, Han Wang, Tingting Zhao, Yangming Guo, Zhuoyang Xu, Zhuo Liu, Siqi Liu, Xiang Lan, Xingzhi Sun and Mengling Feng	Classification of Cardiac Abnormalities From ECG Signals Using SE-ResNet	Link (145MB)
4	Triage	Maximilian Oppelt, Maximilian Riehl, Felix Kemeth and Jan Steffan	Combining Scatter Transform and Deep Neural Networks for Multilabel ECG Signal Classification	Link (79MB)
5	Sharif AI Team	Hosein Hasani, Adeleh Bitarafan and Mahdieh Soleymani	Classification of 12-lead ECG Signals with Adversarial Multi-Source Domain Generalization	Link (25MB)
6	DSAIL_SNU	Seonwoo Min, Hyun-Soo Choi, Hyeongrok Han, Minji Seo, Jin-Kook Kim, Junsang Park, Sunghoon Jung, Il-Young Oh, Byunghan Lee and Sungroh Yoon	Bag of Tricks for Electrocardiogram Classification with Deep Neural Networks	Link (99MB)
7	UMCUVA	Max Bos, Rutger van de Leur, Jeroen Vranken, Deepak Gupta, Pim van der Harst, Pieter Doevendans and René van Es	Automated Comprehensive Interpretation of 12-lead Electrocardiograms Using Pre-trained Exponentially Dilated Causal Convolutional Neural Networks	Link (286MB)
8	CQUPT_ECG	Jiabo Chen, Tianlong Chen, Bin Xiao, Xiuli Bi, Yongchao Wang, Weisheng Li, Han Duan, Junhui Zhang and Xu Ma	SE-ECGNet: Multi-scale SE-Net for Multi-lead ECG Data	Link (221KB)
9	ECU	Najmeh Fayyazifar, Selam Ahderom, David Suter, Andrew Maiorana and Girish dwivedi	Impact of Neural Architecture Design on Cardiac Abnormality Classification Using 12-lead ECG Signals	Link (151KB)
10	PALab	wenxiao jia, Xiao Xu, Xian Xu, Yuyao Sun and Xiaoshuang Liu	Arrhythmia Detection and Classification of 12-lead ECGs Using a Deep Neural Network	Link (402KB)
11	HITTING	Radovan Smisek, Andrea Nemcova, Lucie Marsanova, Lukas Smital, Martin Vítek and Jiri Kozumplik	Cardiac Pathologies Detection and Classification in 12-lead ECG	Link (58MB)
12	Gio_Ivo	giovanni bortolan, Ivaylo Christov and Iana Simova	Rule-Based methods and Deep Learning Networks for Automatic Classification of ECG	Link (87MB)
13	AUTh Team	Charilaos Zisou, Andreas Sochopoulos and Konstantinos Kitsios	Convolutional Recurrent Neural Network and LightGBM Ensemble Model for 12-lead ECG Classification	Link (52MB)
14	BioS	Mateusz Soliński, Michał Łepek, Antonina Pater, Katarzyna Muter, Przemysław Wiszniewski, Dorota Kokosińska, Judyta Salamon and Zuzanna Puzio	12-lead ECG Arrythmia Classification Using Convolutional Neural Network for Mutually Non-Exclusive Classes	Link (58MB)
15	UC_Lab_Kn	Lucas Weber, Maksym Gaiduk, Wilhelm Daniel Scherz and Ralf Seepold	Cardiac Abnormality Detection in 12-lead ECGs with Deep Convolutional Neural Networks Using Data Augmentation	Link (208KB)
16	Cardio-Challengers	Akash Kirodiwal, Apoorva Srivastava, Ashutosh Dash, Ayantika Saha, Gopi Vamsi Penaganti, Sawon Pratiher, sazedul alam, Amit Patra, Nirmalya Ghosh and Nilanjan Banerjee	A Bio-toolkit for Multi-Cardiac Abnormality Diagnosis Using ECG Signal and Deep Learning	Link (987KB)
17	JuJuRock	Po-Ya Hsu, Po-Han Hsu, Tsung-Han Lee and Hsin-Li Liu	Multi-label Arrhythmia Classification From 12-Lead Electrocardiograms	Link (36MB)
18	Minibus	Ran Duan, Xiaodong He and Ouyang Zhuoran	MADNN: A Multi-scale Attention Deep Neural Network for Arrythmia Classification	Link (17MB)
19	Desafinado	Durmus Umutcan Uguz, Felix Berief, Steffen Leonhardt and Christoph Hoog Antink	Classification of 12-lead ECGs Using Gradient Boosting on Features Acquired With Domain-Specific and Domain-Agnostic Methods	Link (255MB)
20	Team UIO	Bjørn-Jostein Singstad and Christian Tronstad	Convolutional Neural Network and Rule-Based Algorithms for Classifying 12-lead ECGs	Link (19KB)
21	Eagles	Andrew Demonbreun and Grace Mirsky	Automated Classification of Electrocardiograms Using Wavelet Analysis and Deep Learning	Link (60MB)
22	BUTTeam	Tomas Vicar, Jakub Hejc, Petra Novotna, Marina Ronzhina and Oto Janousek	ECG Abnormalities Recognition Using Convolutional Network With Global Skip Connections and Custom Loss Function	Link (124MB)
23	DSC	Georgi Nalbantov, Svetoslav Ivanov and Jeffrey van Prehn	Multi-Class Classification of Pathologies Found on Short ECG Signals	Link (5.4MB)
24	Pink Irish Hat	Halla Sigurthorsdottir, Jérôme Van Zaen, Ricard Delgado-Gonzalo and Mathieu Lemay	ECG Classification With a Convolutional Recurrent Neural Network	Link (7.6MB)
25	Madhardmax	Hardik Rajpal, Madalina Sas, Rebecca Joakim, Chris Lockwood, Nicholas S. Peters and Max Falkenberg	Interpretable XGBoost Based Classification of 12-lead ECGs Applying Information Theory Measures From Neuroscience	Link (7.0MB)
26	Care4MyHeart	Mohanad Alkhodari, Leontios J. Hadjileontiadis and Ahsan H. Khandoker	Identification of Cardiac Arrhythmias from 12-lead ECG using Beat-wise Analysis and a Combination of CNN and LSTM	Link (43MB)
27	MCIRCC	Sardar Ansari, Christopher Gillies, Brandon Cummings, Jonathan Motyka, Guan Wang, Kevin Ward and Hamid Ghanbari	Classification of 12-Lead Electrocardiograms Using Residual Neural Networks and Transfer Learning	Link (30MB)
28	heartly-ai	Philipp Sodmann and Marcus Vollmer	ECG Segmentation using a Neural Network as the Basis for Detection of Cardiac Pathologies	Link (42MB)
29	Code Team	Antonio H. Ribeiro, Daniel Gedon, Daniel Martins Teixeira, Manoel Horta Ribeiro, Antonio Luiz Ribeiro, Thomas B. Schön and Wagner Meira Jr	Automatic 12-lead ECG Classification Using a Convolutional Network Ensemble	Link (276KB)
30	ISIBrno	Petr Nejedly, Adam Ivora, Ivo Viscor, Josef Halamek, Pavel Jurak and Filip Plesinger	Utilization of Residual CNN-GRU with Attention Mechanism for Classification of 12-lead ECG	Link (248KB)
31	Alba_W.O.	Marek Żyliński and Gerard Cybulski	Selected Features for Classification of 12-lead ECGs	Link (58MB)
32	AI Strollers	Rohit Pardasani and Navchetan Awasthi	Classification of 12 Lead ECG Signal Using 1D-CNN With Class Dependent Threshold	Link (41MB)
33	ECGLearner	Yingjing Feng and Edward Vigmond	Deep Multi-Label Multi-Instance Classification on 12-Lead ECG	Link (413KB)
34	Leicester-Fox	Zheheng Jiang, Tiago Paggi de Almeida, Fernando Schlindwein, G. André Ng, Huiyu Zhou and Xin Li	Diagnostic of Multiple Cardiac Disorders from 12-lead ECGs Using Graph Convolutional Network Based Multi-label Classification	Link (39MB)
35	deepzx987	Deepankar Nankani, Pallabi Saikia and Rashmi Dutta Baruah	Automatic Concurrent Arrhythmia Classification Using Deep Residual Neural Networks	Link (555KB)
36	CVC	Alexander William Wong, Weijie Sun, Sunil Vasu Kalmady, Padma Kaul and Abram Hindle	Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble	Link (12MB)
37	Cordi-Ak	Paul Samuel Ignacio, Jay-Anne Bulauan and John Rick Manzanares	A Topology Informed Random Forest Classifier for ECG Classification	Link (891KB)
38	MIndS	Marwen Sallem, Adnen Saadaoui, Amina Ghrissi and Vicente Zarzoso	Detection of Cardiac Arrhythmias From Varied Length Multichannel Electrocardiogram Recordings Using Deep Convolutional Neural Networks	Link (47MB)
39	easyG	Martin Baumgartner, Dieter Hayn, Andreas Ziegl, Alphons Eggerth and Günter Schreier	Multi-Stream Deep Neural Network for 12-Lead ECG Classification	Link (399KB)
40	BiSP Lab	Matteo Bodini, Massimo W Rivolta and Roberto Sassi	Classification of 12-lead ECG with an Ensemble Machine Learning Approach	Link (340KB)
41	Technion_AIMLAB	David Assaraf, Jeremy Levy, Janmajay Singh, Armand Chocron and Joachim A. Behar	Classification of 12-lead ECGs using digital biomarkers and representation learning	Link (474KB)

Unranked submissions:

Team name	Team member(s)	Paper	Code
AAIST	—	—	Link (8.3KB)
AImsterdam	—	—	Link (27MB)
BERCLAB UND	—	—	Link (12MB)
BME_Feng	—	—	Link (7.3MB)
BraveHeart400	—	—	Link (147MB)
BRIC	—	—	Link (276KB)
Chapman	—	—	Link (24KB)
Connected_Health	—	—	Link (29KB)
Health team Szeged	—	—	Link (19KB)
IBMTpeakyFinders	—	—	Link (9.0MB)
Kimball_IRL	—	—	Link (27KB)
LaussenLabs	Sebastian Goodfellow, Dmitrii Shubin, Danny Eytan, Andrew Goodwin, Anusha Jega, Azadeh Assadi, Mjaye Mazwi, Robert Greer, Sujay Nagaraj, Peter Laussen, William Dixon and Carson McLean	Rhythm classification of 12-lead ECGs using deep neural network and class-activation maps for improved explainability	Link (13MB)
LIST_AIHealthCare	—	—	Link (26KB)
Marquette	David Kaftan and Richard Povinelli	A Deep Neural Network and Reconstructed Phase Space Approach to Classifying 12-lead ECGs	Link (21MB)
Medics	—	—	Link (58KB)
MetaHeart	Zhengling He, Pengfei Zhang, Lirui Xu, Zhongrui Bai, Hao Zhang, Weisong Li, Pan Xia and Xianxiang Chen	A Novel Convolutional Neural Network for Arrhythmia Detection From 12-lead Electrocardiograms	Link (25MB)
Metformin-121	—	—	Link (16KB)
ML Warriors	—	—	Link (3.6MB)
NACAS_12X	—	—	Link (100MB)
nebula	Wenjie Cai, Shuaicong Hu, Jingying Yang and Jianjian Cao	Automatic 12-lead ECG Classification Using Deep Neural Networks	Link (45MB)
NN-MIH	Naoki Nonaka and Jun Seita	Electrocardiogram Classification by Modified EfficientNet with Data Augmentation	Link (59MB)
NTU-Accesslab	Yu-Cheng Lin, Yun-Chieh Lee, Wen-Chiao Tsai, Win-Ken Beh and An-Yeu Wu	Explainable Deep Neural Network for Identifying Cardiac Abnormalities Using Class Activation Map	Link (339KB)
Orange Peel	—	—	Link (12KB)
SBU_AI	Ibrahim Hammoud, IV Ramakrishnan and Petar Djuric	Classification of 12-lead ECGs Using Intra-Heartbeat Discrete-time Fourier Transform and Inter-Heartbeat Attention	Link (13MB)
SpaceOn Flattop	Shan Yang, Heng Xiang, Qingda Kong and Chunli Wang	Multi-label Classification of Electrocardiogram With Modified Residual Networks	Link (364MB)
try again	—	—	Link (11KB)
UniA4Life	—	—	Link (80KB)

Tables of final scores

The final scores for the 2020 PhysioNet/CinC Challenge are in the following tables:

1. The official scores
2. The unofficial scores
3. The metrics per database of the official entries
4. The per-class scoring metrics of the official entries on the validation data

We introduced a new scoring metric for this Challenge. We used this scoring metric to evaluate and rank the Challenge entries. We included several other metrics for reference. The area under the receiver operating characteristic (AUROC), area under the precision recall curve (AUPRC), and F-measure scores are the macro-average of the scores across all classes. The accuracy metric is the fraction of correctly diagnosed recordings, i.e., all classes for the recording are correct. These metrics were computed by the evaluate_12ECG_score.py script. Please see the script for more details of these scores.

We included the scores on the following datasets:

1. Validation Set: Includes recordings from the hidden CPSC and G12EC sets.
2. Hidden CPSC Set: Split between the validation and test sets.
3. Hidden G12EC Set: Split between the validation and test sets.
4. Hidden Undisclosed Set: All recordings were part of the test sets.
5. Test Set: Includes recordings from the hidden CPSC, G12EC, and undisclosed test sets.

To refer to these tables in a publication, please cite Perez Alday EA, Gu A, Shah AJ, Robichaux C, Wong AI, Liu C, Liu F, Rad AB, Elola A, Seyedi S, Li Q, Sharma A, Clifford GD^*, Reyna MA^*. Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020. Physiol Meas. 41 (2020). doi: 10.1088/1361-6579/abc960.

In these tables, you can find the following information:

1. Official entries that were scored on the validation and test data and ranked in the Challenge: physionet_2020_official_scores.csv
2. Unofficial entries that were scored on the validation and test data but unranked because they did not satisfy all of the rules or were unsuccessful on one or more of the test sets: physionet_2020_unofficial_scores.csv
3. Challenge and other scoring metrics on all official entries broken with scores for each database in the validation and test data: physionet_2020_metrics_perDatabase_official_entries.csv
4. Per-class scoring metrics on the validation data: physionet_2020_validation_metrics_by_class_official_entries.csv

---

Supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB) under NIH grant number R01EB030362.

Back